Release for Python 3.9/3.10 Base Image

Tag to generate a versioned release of the Python 3.9/3.10 base image

Fix small issue in python base image dockerruns

Release Notes

For: uc-cdis/cloud-automation

Notes since tag: pybase3-3.0.1

Notes to tag/commit: b692c0a

Generated: 2021-11-11

New Features

• Added healthcheck script/cronjob for squid autohealing (#1770)
• Using new manifest values for aws gateway (#1781)

Bug Fixes

• [[ -z $THING ]] syntax not available in default sh , change to [ -z $THING ] (#1787)

Improvements

• More documentation around Arborist nginx configuration for access to its
  API (#1782)

Python 3.10 Base Image

Release new version to trigger new image builds

Python 3.10 Base Image

Release Notes

For: uc-cdis/cloud-automation

Notes since tag: pybase3-2.0.1

Notes to tag/commit: cc76344

Generated: 2021-11-08

New Features

• new python 3.10 base image based off 3.6-buster (#1773)

pybase3-2.0.1

Release Notes

For: uc-cdis/cloud-automation

Notes since tag: pybase3-2.0.0

Notes to tag/commit: pybase3-2.0.1

Generated: 2021-11-05

Bug Fixes

• Bug Fix: python nginx buster; change echo -e to printf for portability
  (#1780)

New Python Base Image: Debian-based Python 3.6

IMPORTANT NOTE: The new python3.6-buster Dockerfile is not replacing existing images so you need to make sure to update downstream services to pull the image from a different location in our container registry. It's being built here: quay.io/cdis/python (https://quay.io/repository/cdis/python?tab=builds)

Release Notes

For: uc-cdis/cloud-automation
Notes since tag: pybase3-1.6.2
Notes to tag/commit: 4ab97f0
Generated: 2021-10-28

New Features

• New Python3.6 Debian Buster dockerfile (based heavily on official
  dockerfiles for using Python / UWSGI / NGINX and previous Alpine-based
  dockerfile) (#1753)
• gen3 job for syncing aggregate metadata to a metadata service (#1722)
• Added netnolimit for indexd (#1725)

Bug Fixes

• Fix some DD APM env vars for Hatchery (#1766)
• Added netpolicy for mariner to talk to wts (#1754)
• fix duplicate entries in the nginx access log (#1748)
• skip aws_es_proxy_setup if ConfigMap doesn't exist (#1744)
• Added in netnolimit so DD apm services can talk to Datadog (#1732)
• Removed prometheus from roll all, to get ready to deprecate it (#1720)

Improvements

• Setup DD APM env vars for Guppy (#1766)
• Logic to use new binary path from this PR
  uc-cdis/aws-es-proxy#2 (#1764)
• Increase resources for aws-es-proxy (#1731)
• Increasing the nginx proxy_read_timeout to 300 seconds (#1760)
• Update timeouts to /guppy to 10 mins (#1749)
• Hatchery deployment: change HOSTNAME to GEN3_ENDPOINT (#1740)
• route GA4GH DRS Access requests to more performance Fence endpoint over
  indexd for latest fence image versions (#1726)

increase nginx header limit

Release Notes

For: uc-cdis/cloud-automation

Notes since tag: pybase3-1.6.1

Notes to tag/commit: bc37145

Generated: 2021-09-07

New Features

• Added jenkins-ci cleanup to jenkins cronjob (#1712)
• save-failed-pod-logs now will save both initContainer and container
  logs for specific pod (#1711)
• When sower config contains the "batch-export" job, set up s3 buckets +
  secrets to support this job (#1708)
• Adding .data-commons.org and .va.data-commons.org to squid proxy
  whitelist. (#1709)
• Create configuration scripts for metadata-service (#1701)
• Add DD APM support to Hatchery (#1695)
• Added sqs monitoring (#1677)
• Added little script to update the ssl cipher suite for revproxy (#1678)
• Added slack webhook alerts to standard ETL jobs (#1675)
• Added netpolicies to mariner (#1665)
• distribute-licenses cron job to scan running pods for unlicensed stata
  workspaces requresting a license, copy over demo licenses (#1663)
• Allow connecting to the monqcle API for PDAPS data (#1659)
• allow traffic to fwww.bc.edu, repec.org (ssc repositories) (#1658)
• https://ctds-planx.atlassian.net/browse/HP-239 (#1658)
• Added ability to configure cronjobs through manifest (#1640)
• Squid update to allow clinicaltrials.gov (#1655)
• Enable web access to the ICPSR repository (#1647)

Bug Fixes

• Increased header buffer size to prevent header too large error (#1724)
• Made mariner creds optional so EKS IAM deployments can leverage the
  attached SA instead (#1654)
• Only try to allocate ES proxy on metadata setup if the feature that needs
  it is enabled (#1649)
• Added missing datadog permissions (#1628)

Improvements

• Hatchery: mount HOSTNAME from manifest-global (#1717)
• Move aggregate MDS configuration to our standard manifest config process
  (#1701)
• Add samesite="Lax" to cookies of session, visitor, csrftoken and
  service_releases (#1703)
• MDS: Do not restrict body size for mds (#1706)
• Pass the data upload bucket name in fence-config into portal as
  DATA_UPLOAD_BUCKET when doing kube-setup-portal (#1687)
• Pass Datadog RUM application ID and client token into Portal (#1651)
• using quay images for selenium to prevent failures due to dockerhub rate
  limits (#1652)
• Common code to wait for Elasticsearch to be ready (#1638)

Dependency Updates

• Ruby 2.5 will be EOL 7/30, updating to latest tested ruby version to
  prevent deprecation (#1676)
• Metadata service now uses Elasticsearch for the sake of aggregated metadata
  APIs (#1638)

Deployment Changes

• Remove USE_AGG_MDS and AGG_MDS_NAMESPACE from
  Gen3Secrets/g3auto/metadata/metadata.env and set those variables in a
  manifest: {} block in manifest.json (#1701)
• Migrate the appropriate agg MDS config to the relevant manifest repo under
  a metadata/aggregate_config.json path (#1701)
• Run gen3 kube-setup-metdata and roll the metadata service in Kubernetes
  (#1701)
• Add cronjobs to the manifest by making a cdis-manifest/(commons
  folder)/manifests/cronjobs/cronjobs.json file with each cron name being a
  key and the cron schedule being the value. (#1640)

Dependency Updates

• Bumps lodash from 4.17.19 to 4.17.21.
  (#1600)

Disable prometheus metrics aggregation

Release Notes

For: uc-cdis/cloud-automation
Notes since tag: pybase3-1.6.0
Notes to tag/commit: pybase3-1.6.1
Generated: 2021-06-28

New Features

• Add anaconda.org in addition to anaconda.com to the squid allow list
  (#1641)
• Added qualy agent configuration to userdata bootstrap scripts (#1626)
• New CLI module "gen3 sqs" to manage AWS SQS queues (#1603)
• Setting up the audit service now involves the creation of an AWS SQS (#1603)
• Fence now uses service account "fence-sa" which has access to push messages
  to the audit SQS (#1603)
• The audit service now uses service account "audit-service-sa" which has
  access to read messages in the audit SQS (#1603)

Bug Fixes

• Added bucket permissions to EKS worker node policy (#1637)
• Add initial SA setup for that deployments can schedule pods. (#1632)
• Skipped terraform sqs setup from non-adminvm environments (#1630)

Deployment Changes

• chore(observability): Remove Prometheus metrics aggregation (#1642)
• Add activation id/customer id to EKS module config.tfvars if you want
  qualys agent configured there (#1626)
• The new version of the audit service will require running gen3 kube-setup-audit-service and gen3 kube-setup-fence again, and updating
  the audit-service and fence configuration files (#1603)

Added datadog apm integration to core python services

Release Notes

For: uc-cdis/cloud-automation
Notes since tag: pybase3-1.5.2
Notes to tag/commit: pybase3-1.6.0
Generated: 2021-06-09

• feat(dd-apm): Added initial config for dd apm
• feat(dd-apm): Added datadog apm integration to core python services

More Reliability and security updates

For: uc-cdis/cloud-automation
Notes since tag: pybase3-1.5.1
Notes to tag/commit: pybase3-1.5.2
Generated: 2021-06-07

Bug Fixes

• Fix ENABLE_SVC_METRICS_SCRAPING logic (#1623)