Issue link: https://read.uberflip.com/i/1187300
Amazon Web Services – HPC Lens AWS Well-Architected Framework Page 14 1. A user initiates a cluster using the AWS CLI. 2. AWS CloudFormation executes the cluster architecture as described in a cluster template file, to which the user has contributed a few custom settings (for example, by editing a configuration file or using a web interface). 3. AWS CloudFormation deploys the infrastructure from snapshots created with customized HPC software/applications that cluster instances can access through an NFS export. 4. The user logs into the master node and submits jobs to the scheduler (for example, SGE). 5. The master node emits metrics to CloudWatch based on job queue size. 6. CloudWatch triggers Auto Scaling events to increase the number of compute nodes if the job queue size exceeds a threshold. 7. Scheduled jobs are processed by the compute fleet. 8. Source data and result data are stored in an S3 bucket. 9. Optional EBS snapshots can be taken to preserve changes to the EBS volumes. Serverless The HTC cloud journey often leads to an environment that is entirely serverless, meaning that you can concentrate on your applications and leave the server provisioning to managed services. AWS Lambda lets you run code without the need to provision or manage servers. You pay only for the compute time you consume—there is no charge when your code is not running. You upload your code, and Lambda takes care of everything required to run and scale your code. Lambda also offers the capability of automatically triggering off of events from other AWS services. Scalability is a second advantage of the serverless Lambda approach. Although each worker may be modest in size–say, a compute core with some memory–the architecture can spawn thousands of concurrent Lambda workers, thus reaching a large compute throughput capacity and earning the HPC label. For example, a large number of files can be analyzed by invocations of the same algorithm, a large number of genomes can be analyzed in parallel, or a large number of gene sites within a genome can be modeled. Not only does the largest attainable scale matter, but so does the speed of scaling. While server-based architectures