Tag to generate a versioned release of the Python 3.9/3.10 base image

Release Notes

For: uc-cdis/cloud-automation

Notes since tag: pybase3-3.0.1

Notes to tag/commit: b692c0af2bbc35b71417409c6d580a1587220870

Generated: 2021-11-11

New Features

• Added healthcheck script/cronjob for squid autohealing (#1770)
• Using new manifest values for aws gateway (#1781)

Bug Fixes

• [[ -z $THING ]] syntax not available in default sh , change to [ -z $THING ] (#1787)

Improvements

• More documentation around Arborist nginx configuration for access to its API (#1782)

Release new version to trigger new image builds

Release Notes

For: uc-cdis/cloud-automation

Notes since tag: pybase3-2.0.1

Notes to tag/commit: cc76344e9d925eb57ee60b3c5f6e02b8ea4e9a81

Generated: 2021-11-08

New Features

• new python 3.10 base image based off 3.6-buster (#1773)

Release Notes

For: uc-cdis/cloud-automation

Notes since tag: pybase3-2.0.0

Notes to tag/commit: pybase3-2.0.1

Generated: 2021-11-05

Bug Fixes

• Bug Fix: python nginx buster; change echo -e to printf for portability (#1780)

IMPORTANT NOTE: The new python3.6-buster Dockerfile is not replacing existing images so you need to make sure to update downstream services to pull the image from a different location in our container registry. It's being built here: quay.io/cdis/python (https://quay.io/repository/cdis/python?tab=builds)

Release Notes

For: uc-cdis/cloud-automation Notes since tag: pybase3-1.6.2 Notes to tag/commit: 4ab97f0351d72ddaae3ccb04f8a148c6011051bd Generated: 2021-10-28

New Features

• New Python3.6 Debian Buster dockerfile (based heavily on official dockerfiles for using Python / UWSGI / NGINX and previous Alpine-based dockerfile) (#1753)
• gen3 job for syncing aggregate metadata to a metadata service (#1722)
• Added netnolimit for indexd (#1725)

Bug Fixes

• Fix some DD APM env vars for Hatchery (#1766)
• Added netpolicy for mariner to talk to wts (#1754)
• fix duplicate entries in the nginx access log (#1748)
• skip aws_es_proxy_setup if ConfigMap doesn't exist (#1744)
• Added in netnolimit so DD apm services can talk to Datadog (#1732)
• Removed prometheus from roll all, to get ready to deprecate it (#1720)

Improvements

• Setup DD APM env vars for Guppy (#1766)
• Logic to use new binary path from this PR https://github.com/uc-cdis/aws-es-proxy/pull/2 (#1764)
• Increase resources for aws-es-proxy (#1731)
• Increasing the nginx proxy_read_timeout to 300 seconds (#1760)
• Update timeouts to /guppy to 10 mins (#1749)
• Hatchery deployment: change HOSTNAME to GEN3_ENDPOINT (#1740)
• route GA4GH DRS Access requests to more performance Fence endpoint over indexd for latest fence image versions (#1726)

Release Notes

For: uc-cdis/cloud-automation

Notes since tag: pybase3-1.6.1

Notes to tag/commit: bc37145ec4f05a5a0fc320aff9548ded8fda5ad2

Generated: 2021-09-07

New Features

• Added jenkins-ci cleanup to jenkins cronjob (#1712)
• save-failed-pod-logs now will save both initContainer and container logs for specific pod (#1711)
• When sower config contains the "batch-export" job, set up s3 buckets + secrets to support this job (#1708)
• Adding .data-commons.org and .va.data-commons.org to squid proxy whitelist. (#1709)
• Create configuration scripts for metadata-service (#1701)
• Add DD APM support to Hatchery (#1695)
• Added sqs monitoring (#1677)
• Added little script to update the ssl cipher suite for revproxy (#1678)
• Added slack webhook alerts to standard ETL jobs (#1675)
• Added netpolicies to mariner (#1665)
• distribute-licenses cron job to scan running pods for unlicensed stata workspaces requresting a license, copy over demo licenses (#1663)
• Allow connecting to the monqcle API for PDAPS data (#1659)
• allow traffic to fwww.bc.edu, repec.org (ssc repositories) (#1658)
• https://ctds-planx.atlassian.net/browse/HP-239 (#1658)
• Added ability to configure cronjobs through manifest (#1640)
• Squid update to allow clinicaltrials.gov (#1655)
• Enable web access to the ICPSR repository (#1647)

Bug Fixes

• Increased header buffer size to prevent header too large error (#1724)
• Made mariner creds optional so EKS IAM deployments can leverage the attached SA instead (#1654)
• Only try to allocate ES proxy on metadata setup if the feature that needs it is enabled (#1649)
• Added missing datadog permissions (#1628)

Improvements

• Hatchery: mount HOSTNAME from manifest-global (#1717)
• Move aggregate MDS configuration to our standard manifest config process (#1701)
• Add samesite="Lax" to cookies of session, visitor, csrftoken and service_releases (#1703)
• MDS: Do not restrict body size for mds (#1706)
• Pass the data upload bucket name in fence-config into portal as DATA_UPLOAD_BUCKET when doing kube-setup-portal (#1687)
• Pass Datadog RUM application ID and client token into Portal (#1651)
• using quay images for selenium to prevent failures due to dockerhub rate limits (#1652)
• Common code to wait for Elasticsearch to be ready (#1638)

Dependency Updates

• Ruby 2.5 will be EOL 7/30, updating to latest tested ruby version to prevent deprecation (#1676)
• Metadata service now uses Elasticsearch for the sake of aggregated metadata APIs (#1638)

Deployment Changes

• Remove USE_AGG_MDS and AGG_MDS_NAMESPACE from Gen3Secrets/g3auto/metadata/metadata.env and set those variables in a manifest: {} block in manifest.json (#1701)
• Migrate the appropriate agg MDS config to the relevant manifest repo under a metadata/aggregate_config.json path (#1701)
• Run gen3 kube-setup-metdata and roll the metadata service in Kubernetes (#1701)
• Add cronjobs to the manifest by making a cdis-manifest/(commons folder)/manifests/cronjobs/cronjobs.json file with each cron name being a key and the cron schedule being the value. (#1640)

Dependency Updates

• Bumps lodash from 4.17.19 to 4.17.21. (#1600)

Release Notes

For: uc-cdis/cloud-automation Notes since tag: pybase3-1.6.0 Notes to tag/commit: pybase3-1.6.1 Generated: 2021-06-28

New Features

• Add anaconda.org in addition to anaconda.com to the squid allow list (#1641)
• Added qualy agent configuration to userdata bootstrap scripts (#1626)
• New CLI module "gen3 sqs" to manage AWS SQS queues (#1603)
• Setting up the audit service now involves the creation of an AWS SQS (#1603)
• Fence now uses service account "fence-sa" which has access to push messages to the audit SQS (#1603)
• The audit service now uses service account "audit-service-sa" which has access to read messages in the audit SQS (#1603)

Bug Fixes

• Added bucket permissions to EKS worker node policy (#1637)
• Add initial SA setup for that deployments can schedule pods. (#1632)
• Skipped terraform sqs setup from non-adminvm environments (#1630)

Deployment Changes

• chore(observability): Remove Prometheus metrics aggregation (#1642)
• Add activation id/customer id to EKS module config.tfvars if you want qualys agent configured there (#1626)
• The new version of the audit service will require running gen3 kube-setup-audit-service and gen3 kube-setup-fence again, and updating the audit-service and fence configuration files (#1603)

Release Notes

For: uc-cdis/cloud-automation Notes since tag: pybase3-1.5.2 Notes to tag/commit: pybase3-1.6.0 Generated: 2021-06-09

• feat(dd-apm): Added initial config for dd apm
• feat(dd-apm): Added datadog apm integration to core python services

For: uc-cdis/cloud-automation Notes since tag: pybase3-1.5.1 Notes to tag/commit: pybase3-1.5.2 Generated: 2021-06-07

Bug Fixes

• Fix ENABLE_SVC_METRICS_SCRAPING logic (#1623)