Life Sciences

Genomics Data Transfer, Analytics, and Machine Learning using AWS Services AWS Whitepaper Reference architecture From time to time a secondary analysis job will fail. AWS Step Functions has support for job retries and try catch logic to handle job failures with custom code. To learn more about handling secondary analysis task errors or failures, see Handling secondary analysis task errors and workflow failures (p. 24). Create an operational dashboard to monitor the status of your secondary analysis workflow. AWS Step Functions and AWS Batch are both integrated with Amazon CloudWatch so creating a secondary analysis dashboard in Amazon CloudWatch is easy to do. To learn more about creating a secondary analysis operational dashboard in Amazon CloudWatch, see Monitoring secondary analysis workflow status, cost, and performance (p. 25). Note To access an AWS Solutions Implementation providing an AWS CloudFormation template to automate the deployment of the solution in the AWS Cloud, see the Genomics Secondary Analysis Using AWS Step Functions and AWS Batch Implementation Guide. 8

• Genomics Data Transfer, Analytics, and Machine Learning using AWS Services
• Table of Contents
• Genomics Data Transfer, Analytics, and Machine Learning using AWS Services (1)
• Abstract
• Introduction
• Transferring genomics data to the Cloud and establishing data access patterns using AWS DataSync and AWS Storage Gateway for files
• Recommendations
• Reference architecture
• File-based access to Amazon S3
• Running secondary analysis workflows using AWS Step Functions and AWS Batch
• Recommendations
• Reference architecture
• Performing tertiary analysis with data lakes using AWS Glue and Amazon Athena
• Recommendations
• Reference architecture
• Performing tertiary analysis with machine learning using Amazon SageMaker
• Recommendations
• Reference architecture
• Conclusion
• Appendix A: Genomics report pipeline reference architecture
• Appendix B: Research data lake ingestion pipeline reference architecture
• Appendix C: Genomics data transfer, analytics, and machine learning reference architecture
• Appendix D: Compliance resources
• Appendix E: Optimizing data transfer, cost, and performance
• Appendix F: Optimizing storage cost and data lifecycle management
• Appendix G: Optimizing secondary analysis compute cost
• Appendix H: Handling secondary analysis task errors and workflow failures
• Appendix I: Monitoring secondary analysis workflow status, cost, and performance
• Appendix J: Scaling secondary analysis
• Appendix K: Optimizing the performance of data lake queries
• Appendix L: Optimizing the cost of data lake queries
• Contributors
• Document revisions
• Notices