Aws Genomics Workflows Versions Save

Genomics Workflows on AWS

v3.1.0

2 years ago

What's Changed

New Contributors

Full Changelog: https://github.com/aws-samples/aws-genomics-workflows/compare/v3.0.7...v3.1.0

v3.0.7

3 years ago

Bug fixes

  • #159 - add server timeout config for cromwell, specify default region in cromwell server instance

Improvements

  • #138, #154, #157 - add CDK constructs
  • #152 - add custom deployment instructions to docs
  • #156 - add a nextflow+gwfcore combined template

Known Issues

  • #141 - Globbing error staging input data, causing Step Functions workflows to fail

v3.0.6.1

3 years ago

Hotfix

  • fix site deployment

v3.0.6

3 years ago

Bug fixes

  • #144 - run get_cromwell_tools when provisioning cromwell server

Updates

  • Use nextflow image from AWS ECR Public Gallery

Improvements

  • #147 - improve deployment scripts for custom distributions
    • add verbosity control
    • add public asset control
  • #143 - get cromwell.jar from s3, allow cromwell access to user defined private buckets

Known issues

  • #141 - Globbing error staging input data, causing Step Functions workflows to fail

v3.0.5

3 years ago

Bug fixes

  • #119 - Move files from tmp dir on Cromwell server
  • #131 - Enable 4GB swap on instances
  • #134 , #136 - Use "main" branch name for CodeCommit repos
  • #135 - Use aws-cli shim for Cromwell

Updates

  • Use Amazon Linux 2 for instances

Improvements

  • #127 - Send instance provisioning logs to CloudWatch
  • Use ECR public repos for container base images

Known issues

  • #141 - Globbing error staging input data, causing Step Functions workflows to fail

v3.0.4

3 years ago

Bug fixes

  • #128 , #132 - fixed issue running AWS CLI v2 with container images that lack glibc

Updates

  • #125 - fix README instructions for self-deployment, add instructions to access Cromwell server via Amazon Systems Manager.

Improvements

  • #126 - add womtool and cromshell to Cromwell server instance

v3.0.3

3 years ago

Bug fixes

  • #121 - Add SSM policy to Cromwell Server EC2 instance
  • #116 - Change Nextflow example to use demo RNASeq workflow from nextflow-io

Improvements

  • #123 - Prefer using cached container instances to reduce container image pull traffic

v3.0.2

3 years ago

Bug fixes

  • use lower-case stack names when launching from guide for s3 bucket naming compatibility as these are potentially used for namespaces
  • fix s3 http url allowed pattern for template root url parameters
  • pin "test" as a version for e2e testing

Improvements

  • Check for dependencies in _scripts/make_dist

v3.0.1

3 years ago

Updates

  • Add check for aws-cli installation and termination if not available
  • Enable ECS SPOT instance draining

v3.0.0

3 years ago

NOTE: This release provides an improved deployment experience and extensibility as a trade-off to breaking changes with prior 2.x releases.

New

  • A generalized GWFCore that is agnostic of workflow orchestrator and can be used with multiple orchestrators at once.

    • Uses AWS Systems Manager Parameter Store for GWFCore attributes like S3 Bucket paths and AWS Batch Queue Names
    • Project namespacing for multi-tenent deployment
    • Deployments are self-contained - distributed artifacts (e.g. Amazon EBS Autoscale) are stored in S3 on initial deployment
    • Launch template based provisioning can be updated without rebuilding AWS Batch Compute Environments
      • Uses provisioning scripts stored in AWS CodeCommit with changes automatically deployed to S3 via AWS CodeBuild and AWS CodePipeline
    • AWS Batch Compute Environments uses 5th generation C, M, R instance families for cost optimization and limits max instance sizes to 4xl to reduce impact of SPOT terminations

  • Updated Cromwell architecture

    • Compatible with Cromwell v52+
    • Includes Aurora Serveless database for call-caching metadata store
    • Cromwell server instance updated to Amazon Linux 2

  • Generalized container builds for nextflow and Step Functions demo workflow containers

    • Container Dockerfiles stored in AWS CodeCommit
    • Container image builds via AWS CodeBuild

  • Tagged builds are version pinned. Templates and assets are accessible via https://<bucketname>.s3.amazonaws.com/<version>/<assetpath> with <version> being vX.Y.Z or latest. Assets are no longer available at the root bucket level.

Minor fixes and improvements

  • Nextflow container uses upstream base image for version pinning
  • Nextflow container entrypoint captures AWS Batch job termination and gracefully terminates workflow
  • Consistent naming for orchestrator templates

Documentation changes

  • AIO templates removed. These are no longer needed as the GWFCore is generalized and compatible with all documented orchestrators. Recommended installation procedure is to 1) Create or select a VPC with at least 2 public and 2 private subnets; 2) Deploy GWFCore; 3) Deploy desired orchestrators.

  • Add Cromwell and Nextflow "Troubleshooting" documentation

  • Add Cost Optimization documentation