Life Sciences

HPC Lens for the AWS Well-Architected Framework

Issue link: https://read.uberflip.com/i/1187300

Contents of this Issue

Navigation

Page 19 of 46

Amazon Web Services – HPC Lens AWS Well-Architected Framework Page 16 Tightly Coupled, High-Performance Computing Tightly coupled HPC applications consist of parallel processes that are dependent on each other to carry out the calculation. Unlike an HTC simulation, all processes of a tightly coupled HPC simulation iterate together. An iteration is defined as one step of the overall simulation. HPC calculations generally rely on tens to thousands of processes or cores over one to millions of iterations. The failure of one node usually leads to the failure of the entire calculation. To mitigate the risk of complete failure, checkpointing regularly occurs during a simulation to allow for the restarting of a case. HPC cases typically rely on a Message Processing Interface (MPI) for inter- process communication. Shared Memory Parallelism via OpenMP can be used in conjunction with MPI. Examples of tightly coupled HPC workloads include computational fluid dynamics, weather prediction, and reservoir simulation. A suitable architecture for a tightly coupled HPC workload has the following considerations: • Network: The network requirements for tightly coupled calculations are generally very demanding. Slow communication between nodes typically results in the slowdown of the entire calculation. The largest instance size, enhanced networking, and placement groups are required for consistent networking performance. Tightly coupled applications range in size. A large problem size, spread over a large number of processes or cores, usually parallelizes well. Small cases, with lower total computational requirements, place the greatest demand on the network. • Storage: Like HTC workloads, the storage requirements for tightly coupled workloads vary, driven by dataset size and the performance requirements for reading and writing data. A shared file system is often used, either from an NFS export on an instance with an EBS volume, Amazon Elastic File System (Amazon EFS) file system, or a high- performance file system. High-performance file systems can be obtained either from a third party in the AWS Marketplace or can be installed by the user. • Compute: EC2 instances are offered in a variety of configurations with varying core to memory ratios. For parallel applications, it is helpful to spread memory-intensive parallel simulations across more compute

Articles in this issue

view archives of Life Sciences - HPC Lens for the AWS Well-Architected Framework