Concourse Versions Save

Concourse is a container-based continuous thing-doer written in Go.

v7.11.2

1 month ago

🚨 Security

  • fix(deps): update module github.com/opencontainers/runc to v1.1.12 [security] (#8900) @renovate :link:

🀷 Miscellaneous

  • Rotate dev vault certs (#8904) @xtremerui :link:

  • Rebase master 7.11.2 (#8909) @xtremerui :link:

πŸ“¦ Bundled resource types

v7.11.1

2 months ago

✈️ Features

  • add shared path to SSM parameters (#8687) @konstl000 :link:
    • Added `--aws-ssm-shared-path` to configure shared secret paths for AWS SSM cred manager similarly to the one for Vault.

🀷 Miscellaneous

  • Fix incorrect log message (#8865) @hongkuancn :link:

  • Use stable website for internet test in watsjs (#8869) @xtremerui :link:

  • Pulling go version other than relying on runner image in CodeQL scan (#8879) @xtremerui :link:

  • fix(deps): update module github.com/containerd/containerd to v1.7.11 [security] (#8872) @renovate :link:

  • fix(deps): update module github.com/go-jose/go-jose/v3 to v3.0.1 [security] (#8873) @renovate :link:

  • fix(deps): update module golang.org/x/crypto to v0.17.0 [security] (#8874) @renovate :link:

  • fix(deps): update all dependencies (#8875 #8876 #8877 #8878 #8880 #8882 #8884 #8887 #8890) @renovate :link:

  • Fix compilation error in topgun/k8s test (#8889) @xtremerui :link:

πŸ“¦ Bundled resource types

v7.11.0

4 months ago

🚨 Breaking

✈️ Features

  • Make cc.xml endpoint public, and only list public pipelines (#8809) @LukeWinikates :link:

    • Public pipelines are now accessible through the cc.xml endpoint while unauthenticated
  • Emitting "latest_completed_build_status" gauge from prometheus (#8826) @wayneadams :link:

    • Add concourse_builds_latest_completed_build_status metric
      • Guage = 0 for success
      • Guage = 1 for failure
      • Guage = 2 for aborted
      • Guage = 3 for error
  • Add additional help context for metric (#8839) @wayneadams :link:

🐞 Bug Fixes

  • Fixes cf authentication fails on 7.9.1 #8696 (#8806) @wayneadams :link:

    • Fix CF connector regression bug introduced on 7.9.1
  • Fix fly builds cmd with --team flag (#8841) @xtremerui :link:

    • Fix a bug of fly builds command that shows pipeline/job not found when both --team and --pipeline/--job are provided.

🀷 Miscellaneous

  • Update all dependencies (#8789, #8815, #8819, #8821, #8823, #8825, #8830, #8835) @renovate :link:

  • Fix dropped test error in topgun/k8s (#8795) @alrs :link:

  • Fix 404 links to blog posts (#8799) @abjorck :link:

  • Update javascript (#8802, #8831) @renovate :link:

  • Bump imdario/mergo to v1.0.0 (#8810) @taylorsilva :link:

  • Bump concourse/retryhttp to v1.2.4 (#8811) @taylorsilva :link:

  • Bump concourse/flag to v2.0.2 (#8812) @taylorsilva :link:

  • Bump txn2/txeh to v1.5.4 (#8813) @taylorsilva :link:

  • Fix data race in emmiter and pool unit tests (#8832) @xtremerui :link:

    • Fix data race observed in unit tests for emitter new-relic and worker pool tests.
  • Fix integration flaky ops parallel upgrade/downgrade tests (#8834) @xtremerui :link:

  • Fix integration flaky ops test (#8838) @xtremerui :link:

πŸ“¦ Bundled resource types

v7.10.0

7 months ago

🚨 Breaking

  • cf resource is not included in Concourse binary anymore since its repo has been moved to cloudfoundry community and no longer being maintained by Concourse team.

✈️ Features

  • Update base image of all built-in resource types:

  • Update references to use ginkgo/v2 (#8550) @dtimm :link:

  • Support "raw" encoding for volume streaming. (#8706) @evanchaoli :link:
    Add a new compression method raw to CONCOURSE_STREAMING_ARTIFACTS_COMPRESSION. The new method will cost more network bandwidth of workers but save a lot of worker's CPU times, and make volume streaming dramatically faster. The bigger volume under streaming the more dramatic improvement on streaming speed.

  • Add a drift based number of goroutines to component scheduler. (#8709) @evanchaoli :link:
    Add a new ATC option --num-goroutine-threshold to specify a threshold of goroutine count. If set, when a ATC's goroutine count reaches to threshold, then it will get less possibility to run workloads than other ATCs that have less goroutines. This option will help distribute workloads across ATCs evenly.

  • Hermetic for task container (#8713) @xtremerui :link:

    • add Hermetic: bool to task step configuration. When set to true, the task container will be running without external network access. Only worker runtime containerd supports this feature. There will be a reminder as warning when setting a pipeline contains task step that sets hermetic: true.
  • Optimize db notify. (#8736) @evanchaoli :link:
    Optimized the database notifications, which will reduce TPS/QPS in the database side. A new ATC option --db-notification-bus-queue-size is added, defaults to 10000. If the UI doesn't load logs of running builds in time, then consider to increase value of the option.

  • Added a maximum volume size that can be streamed (#8756) @evanchaoli :link:
    Add a new ATC option `CONCOURSE_STREAMING_SIZE_LIMITATION" that restricts maximum size in MB of volumes can be streamed between workers. This is a mechanism to prevent rogue pipeline from hurting multiple workers.

🐞 Bug Fixes

  • atc/db: fix dropped error (#8678) @alrs :link:

  • Fix cf connector error during web node startup (#8699) @xtremerui :link:

    • Fix web node start up error when cf connector is configured
  • Fixed a race condition in component factory. (#8746) @evanchaoli :link:

  • Bump ifrit to fix ATC gracefully terminate issue. (#8751) @evanchaoli :link:

    • Fixed an ATC gracefully terminate issue described in #8747.
  • Add reset character in WaitingForStreamedVolume event render (#8768) @selzoc :link:

  • Unhide the --instance-var option in fly set-pipeline (#8778) @neilmayhew :link:

🀷 Miscellaneous

  • Bump dex to latest (#8666) @xtremerui :link:

  • Fix failed fly integration test in darwin (#8681) @xtremerui :link:

    • Bump Golang to v1.20
  • Update module github.com/containerd/containerd to v1.6.18 [SECURITY] (#8688) @renovate :link:

  • Ignore elm and client-go in renovate deps bump (#8704) @xtremerui :link:

  • bump lager to v3 (#8707) @xtremerui :link:

    • bump code.cloudfoundry.org/lager, concourse/retryhttp and concourse/flag to latest to remove indirect import of ginkgo v1 in Concourse's go.mod file.
  • fix(deps): update module github.com/opencontainers/runc to v1.1.5 [security] (#8718) @renovate :link:

  • fix ginkgo warning and k8s topgun failure (#8723) @xtremerui :link:

  • add events logging when pod is not running for k8s topgun (#8733) @xtremerui :link:

    • Add method in k8s topgun test to log pod events when it is being initialized.
  • Increase timeout for bosh topgun (#8740) @xtremerui :link:

  • Fix test failure due to mock resource that built with paketo jammy (#8760) @xtremerui :link:

  • Remove btrfs baggageclaim test over COS image (#8766) @xtremerui :link:

  • fix(deps): update module github.com/opencontainers/runc to v1.1.5 [security] (#8770) @renovate :link:

  • bumping containerd runtime libs (#8771) @xtremerui :link:

  • refactor: move from io/ioutil to io and os packages (#8774) @Juneezee :link:

  • chore: unnecessary use of fmt.Sprintf or fmt.Sprint (#8786) @testwill :link:

πŸ“¦ Bundled resource types

v7.9.1

1 year ago

✈️ Features

  • Add seccomp profile, hooks dir override for containerd (#8044) @drahnr :link:
    • Adds a worker cli option to override the seccomp filter
    • Adds a worker containerd cli option to pass on a oci hooks dir, for i.e. nvidia gpu mapping

🐞 Bug Fixes

  • Fixed a bug where invalidated worker resource caches are not GC-ed (#8486) @evanchaoli :link:

🀷 Miscellaneous

  • Update all dependencies (#8518) @renovate :link:

  • Update javasript (#8542) @renovate :link:

  • Update module github.com/containerd/containerd to v1.6.12 [SECURITY] (#8642) @renovate :link:

  • Bump dex to latest for security patch (#8644) @xtremerui :link:

  • Security golang dep bumps (#8665) @xtremerui :link:

  • Fix baggageclaim and container limit tests in k8s-topgun (#8670) @xtremerui :link:

  • Fix failed fly integration test in darwin for release/7.9.x (#8682) @xtremerui :link:

πŸ“¦ Bundled resource types

v7.9.0

1 year ago

🚨 Breaking

  • Fix DB out of range error due to build numbers exceed the integer limit (#8390) @xtremerui :link:

    • To allow the migration to run Postgresql version has to be v11+. It happens to be a good timing to drop support of Postgresql v9.6.
  • Fixed a bug of leaking resource cofig scope ids. (#8620) @evanchaoli :link:

    • When global-resources is enabled, resource_config_scopes tables leaked IDs. A side effect of the bug is that unnecessary insert will be performed (see #8618 for details). So, this PR will fix the ID leaking problem and improve performance also.
    • When global-resources is enabled, old resources weren't affected. This fix ensures old resources to switch to global scopes.

    BREAKING: With this change, when switching global-resources from OFF to ON, all resource histories will be lost. It is equivalent to changing source of a resource and causing version history to be lost. Depending on a resource's check behavior, versions may be regenerated.

    If your deployment has turned ON global-resources before the upgrade, or you choose to stay with global-resources OFF, this "breaking" change won't impact your deployment.

    If you upgrade to this version then turn ON global-resources, as described, version histories will lost. You can turn OFF global-resources again and old version histories should come back.

    Note that, if your cluster has turned ON global-resources, and you plan to turn it OFF, no matter what version it is, after turning OFF global-resources, each resource will have an unique version history, thus shared version history will be lost. The behaviour comes with global-resources and it has nothing to do with this change.

✈️ Features

  • Bump dependencies for worker runtime to support Ubuntu Jammy Jellyfish

    • Noted, guardian runtime is still under development to fully support Ubuntu Jammy. In fact, it does not work on any linux distribution with cgroups v2 enabled. We decided to bump the dependencies still for users who want to use latest linux distribution and willing to tweak their OS to enable cgroups v1.
  • load_var step supported var interpolation for file and format (#8387) @evanchaoli :link:

  • Enhancement of component scheduling so that workloads are distributed across ATCs more evenly (#8463) @evanchaoli :link:

  • Turn off connection tracker by default and provide an option to turn on. (#8480) @evanchaoli :link:

    • Disable /debug/connections at ATC start time. It can be enabled at runtime by /debug/connections/on or be disabled by /debug/connections/off again.
  • Enhance Vault API client to auto retry upon rate limit. (#8481) @evanchaoli :link:

    • Enhanced Vault credential manager to auto retry when hitting Vault rate limit error. Vault started to support rate limit since 1.5. When setting rate limit on Vault, it's better to enable rate limit HTTP response header by vault write sys/quotas/config enable_rate_limit_response_headers=true, so that the response header Retry-After may guide the Vault API client to retry after a reasonable duration.
  • Remove "check build started" and "check build finished" metrics (#8485) @evanchaoli :link:

    • To monitor checks, use "check started" and "check finished" instead.
  • Support a way to skip implied get after put. (#8492) @evanchaoli :link:

    • Added no_get option to put step to skip implied get. For example:
      - put: email
        no_get: true
        params:
          ...
      
  • Add --check-container-placement-strategy. (#8494) @evanchaoli :link:

  • Explicit error messeges in p2p streaming (#8559) @dhantha :link:

  • New pipelines without build should be paused automatically with a configurable interval. (#8577) @SimonXming :link:

    • Using params pause-pipelines-after, so pipelines could be paused automatically with configurable interval.
  • Stream: Adding new NextEventRaw method (#8588) @gaelL :link:

  • Change id of table resource_config_scopes to bigint (#8606) @evanchaoli :link:
    Convert id column of the resource_config_scopes table and all tables referencing resource_config_scope_id to a bigint.

  • Performance optimize on accessor. (#8613) @evanchaoli :link:
    Optimized performance of the login authentication process, which will benefit large deployments that has a lot teams and a lot of UI/fly accesses.

🐞 Bug Fixes

  • Since v7.4.0, the Concourse linux tarball in attached binaries has been using the ubuntu version with size 1GB+. Refer to this CI fix for details. Now the linux tarball is set to the version with alpine based resource types again.

  • Add tooltip to username if overflow (#8341) @xtremerui :link:

    • When username is overflowing, show a hovering tooltip with full name in web UI so it won't block buttons below it e.g. trigger build buttons in build page.
  • Fix step header key value UI in build page (#8406) @xtremerui :link:

    • Fix line height of step header in build page when there is sub header like instance vars or across
  • Fixed a bug of error invalidated-worker-resource-cache-exists (#8416) @evanchaoli :link:

  • Add missing lock metrics ResourceGet and VolumeStreaming. (#8468) @evanchaoli :link:

  • Check build should not auto retry. (#8493) @evanchaoli :link:

    • If a check happens to drop into endless retry, there is no way to abort a check build.
  • Fix a bad SQL for check gc. (#8500) @evanchaoli :link:

    • Optimized performance of check-build-events collector.
  • Use pq.Array to avoid hitting parameter limits (#8528) @ae-govau :link:

  • Change host to event_host tag for Datadog integration (#8544) @pablokbs :link:

  • Fix bug in testflight suite env var assignment (#8594) @elliot-gould :link:

    • Now it should allow users to use environment variables to override local user credentials properly.
  • Fix across step states bug (#8634) @xtremerui :link:

    • Fix a bug where sub step of across step showing incorrect state.

🀷 Miscellaneous

  • atc/db/lock: preallocate memory (#8584) @florianl :link:

  • Update k8s-topgun configure for external postgresql by pg v11 chart (#8400) @xtremerui :link:

  • Don’t use β€˜here’ as link text (#8467) @quis :link:

  • Rotate vault certs for dev (#8495) @xtremerui :link:

  • Fix json syntax error to enable Renovate bot (#8506) @xtremerui :link:

  • fix(deps): update all dependencies (#8507) @renovate :link:

  • chore(deps): pin dependencies (#8510) @renovate :link:

  • Add resource check before smoke tests (#8546) @xtremerui :link:

  • Update path for example pipeline (#8598) @jjshanks :link:

  • Remove rerun_of int->bigint migrations (#8626) @xtremerui :link:

πŸ“¦ Bundled resource types

v6.8.0

1 year ago

🚨 Breaking

  • If guardian runtime is enabled in your Concourse deployment, do not upgrade to this version as the latest library of guardian has backward compatibility issue that might not work in Ubuntu 18.04 or 20.04.

✈️ Features

  • Bump dependencies for worker runtime to support Ubuntu Jammy Jellyfish by @xtremerui in https://github.com/concourse/concourse/pull/8609
    • Noted, guardian runtime is still under development to fully support Ubuntu Jammy. In fact, it does not work on any linux distribution with cgroups v2 enabled. We decided to bump the dependencies still for users who want to use latest linux distribution and willing to tweak their OS to enable cgroups v1.

🀷 Miscellaneous

πŸ“¦ Bundled resource types

v7.8.3

1 year ago

:link: security

  • Fix team name overwritten bug

    • All Concourse versions prior to v7.8.3 is vulnerable to parameter pollution that allows authorization bypass in functionality that is meant to restrict cross team actions. An user in any team could make certain http requests to trigger unauthorized activity for other teams like pausing pipelines, re-triggering builds or exposing pipelines. (#8580 )
  • Bump Dex to v2.35.1 for CVE-2022-39222. (#8579 )

πŸ“¦ Bundled resource types

v6.7.9

1 year ago

:link: security

  • Fix team name overwritten bug

    • All Concourse versions prior to v6.7.9 is vulnerable to parameter pollution that allows authorization bypass in functionality that is meant to restrict cross team actions. An user in any team could make certain http requests to trigger unauthorized activity for other teams like pausing pipelines, re-triggering builds or exposing pipelines. (#8581)
  • Bump Dex to v2.35.1 for CVE-2022-39222. (#8582)

πŸ“¦ Bundled resource types

v7.8.2

1 year ago

✈️ Features

  • Disable connection tracker by default and provide an option to enable. (#8433) @evanchaoli :link:

    • Disable /debug/connections at ATC start time. It can be enabled at runtime by /debug/connections/on or be disabled by /debug/connections/off again.
  • Add a drift to component interval. (#8453) @evanchaoli :link:

    • Enhancement of component scheduling so that workloads are distributed across ATCs more evenly.
  • Enhance Vault API client to auto retry upon rate limit. (#8461) @evanchaoli :link:

    • Enhanced Vault credential manager to auto retry when hitting Vault rate limit error. Vault started to support rate limit since 1.5. When setting rate limit on Vault, it's better to enable rate limit HTTP response header by vault write sys/quotas/config enable_rate_limit_response_headers=true, so that the response header Retry-After may guide the Vault API client to retry after a reasonable duration.

🐞 Bug Fixes

  • Add missed lock metrics : "ResourceGet" and "VolumeStreaming" (#8460) @evanchaoli :link:

πŸ“¦ Bundled resource types