Buildkite Agent Versions Save

The Buildkite Agent is an open-source toolkit written in Go for securely running build jobs on any device or network

v3.54.0

8 months ago

v3.54.0 (2023-09-05)

Full Changelog

⚠️ We're adjusting how the set of supported OS versions changes over time. For the details, see #2354.

Added

  • New experiment use-zzglob: uses a different library for resolving glob patterns in buildkite-agent artifact upload #2341 (@DrJosh9000)

Changed

  • Logged errors might look different: errors passed back up to main.go from clicommand #2347 (@triarius)
  • HEAD commit found faster: git log is now used to get commit information instead of git show #2323 (@leakingtapan)

Internal

  • Adapt Olfactor to allow sniffing for multiple smells #2332 (@triarius)

v3.53.0

8 months ago

v3.53.0 (2023-08-31)

Full Changelog

Added

  • Artifact upload and download to/from Azure Blob Storage #2318 (@DrJosh9000)

Fixed

  • Fix detection of missing commits on checkout #2322 (@goodspark)
  • [Experimental] Handle the case when unmarshalling a step where there aren't any plugins #2321 (@moskyb)
  • [Experimental] Fix signature mismatches when steps have plugins #2339, #2319 (@DrJosh9000)
  • [Experimental] Catch step env/job env edge case #2340 (@DrJosh9000)

Changed

  • Retry fork/exec errors when running hook #2325 (@triarius)

Internal

  • Fix ECR authentication failure #2337, #2335, #2334 (@DrJosh9000)
  • Split checkout, artifact, and plugin phases out of executor.go #2324 (@triarius)
  • Store experiments in contexts #2316 (@DrJosh9000)

v3.52.1

8 months ago

v3.52.1 (2023-08-23)

Full Changelog

Fixed

  • Fix missing group interpolation #2303 (@DrJosh9000)
  • Experimental fix for agent workers reading plugin directories while they are being written to by other agent workers #2301 (@triarius)

Internal

  • Rework method of pushing releases to RPM repos #2315 #2314 #2312 #2310 #2304 (@DrJosh9000)
  • Update help text with suggestions from docs code review #2313 (@triarius)
  • Fix a flaky shell test #2311 (@triarius)
  • Adjust cli help output to work better with documentation generation #2317 (@triarius)

v3.52.0

9 months ago

v3.52.0 (2023-08-17)

Full Changelog

Note: the buildkite-agent step get command now prints a new line to stdout in circumstances where it previously did not. We advise you to always strip whitespace from the output of this (and other commands). If you are processing the output of this in a POSIX compliant shell with command substitution (i.e. step="$(buildkite-agent step get ...)"), trailing newlines will be removed automatically.

Added

  • [Experimental] Include pipeline and step env in step signatures #2295 (@DrJosh9000)

Fixed

  • Fix step get is printing the address of the stdout stream at the start #2299 (@triarius)

Changed

  • Add a newline after printing errors from the config parser #2296 (@triarius)

Internal

  • Enable mount-buildkite-agent in release pipeline containers #2298 (@DrJosh9000)
  • Update ecr, docker plugins, and agent image ver #2297 (@DrJosh9000)
  • Pin bk cli used in agent pipeline to a commit #2294 (@triarius)

v3.51.0

9 months ago

Known Issues

⚠️ There is a known issue with buildkite-agent step get. This is fixed in v3.52.0

v3.51.0 (2023-08-15)

Full Changelog

Added

  • Add --strict-single-hooks #2268 (@DrJosh9000)
  • Add missing 'an' in annotation help #2285 (@mdb)
  • [Experimental] Verify step signatures #2210 (@moskyb)
  • [Experimental] Pipeline Signing/Verification with JWS #2252 (@moskyb)
  • [Experimental] Include plugins in command step signatures #2292 (@DrJosh9000)

Changed

  • Make the agent send a SIGTERM (configurable) before a SIGKILL to subprocesses #2250 (@triarius)
  • Limit job log length #2192 (@DrJosh9000)
  • Refactor redactor into streaming replacer and use it to redact secrets #2277 (@DrJosh9000)
  • Dependency upgrades #2278 #2274 #2271 #2272 #2275 #2266 (@dependabot[bot])

Fixed

  • Fix fatal: bad object not detected from git fetch #2286 (@triarius)
  • Fix scalar plugin parsing #2264 (@DrJosh9000)

Internal

  • Reorganise step types among files #2267 (@DrJosh9000)
  • Upload test coverage #2270 (@DrJosh9000)
  • Remove unwrapping in error Is methods #2269 (@triarius)
  • Use capacity hint in concat #2288 (@DrJosh9000)
  • Add ordered.Unmarshal, and use it in pipeline parsing #2279 (@DrJosh9000)
  • Create a setup method for config and logger to reduce boilerplate #2281 (@triarius)
  • Add retry for publishing RPMs #2280 (@triarius)
  • Fix data race in testAgentEndpoint #2265 (@DrJosh9000)
  • Fix missing "fmt" import #2287 (@DrJosh9000)

v3.50.4

9 months ago

v3.50.4 (2023-07-31)

Full Changelog

This point release fixes more parser cases in previous point releases of 3.50. We recommend anyone using v3.50.0 through v3.50.3 upgrade, especially when experiencing problems with previously-working pipeline YAML.

  • Legacy step forms (where the step type is specified with type) should parse correctly
  • Scalar (i.e. non-map) block steps should parse correctly
  • Explicit env: null should parse correctly
  • Parser errors should no longer cause the pipeline to be missing steps; either errors should halt processing or unknown step types remain in the output (though these might still be rejected by buildkite.com).

⚠️ There is a remaining known issue concerning environment variable substitution in pipelines. When environment variables that are defined as null at the top level of a pipeline upload are substituted, they will no longer fall back to values defined in the environment of the buildkite-agent. For example, if you had a pipeline

env:
  FOO: null
steps:
- command: echo "FOO=$FOO"

and upload the pipeline as

FOO=fromJob buildkite-agent pipeline upload

the uploaded pipeline was previously equivalent to:

env:
  FOO: null
steps:
- command: echo "FOO=fromJob"

but now, the null value will result in an empty string being substituted instead.

To work around this, we recommend you defined the fallback manually like so:

env:
  FOO: ${FOO:-}
steps:
- command: echo "FOO=$FOO"

Fixed

  • Even More Pipeline Parsing Fixes #2253 (@moskyb)
  • Fix missing return statements when unmarshalling fails #2245 (@moskyb), #2257 (@DrJosh9000)
  • Add future-proofing UnknownStep type #2254 (@DrJosh9000)
  • Nil handling fixes, particularly parsing env: null #2260 (@DrJosh9000)

Changed

  • Remove docker-compose v1 from ubuntu 22.04 and replace with compatibility script #2248 (@triarius)
  • Authentication failure errors when using S3 now mention BUILDKITE_S3_PROFILE and AWS_PROFILE #2247 (@DrJosh9000)

Internal

  • Remove a double check for the existence of a local hook and log when it is missing in debug #2249 (@triarius)
  • Refactor some code in process.go #2251 (@triarius)
  • Store GOCACHE outside container #2256 (@DrJosh9000)
  • Get mime types from github, rather than Apache's SVN Server #2255 (@moskyb)
  • Check that go.mod is tidy in CI #2246 (@moskyb) and fix flakiness of this check #2261 (@triarius)

v3.50.3

9 months ago

v3.50.3 (2023-07-24)

Full Changelog

This release contains a known issue:

Severity Description Fixed in
Medium Several forms of pipelines fail to parse and can no longer be uploaded ✅ Fixed in v3.50.4

Changed

  • Two-phase pipeline parsing #2238 (@DrJosh9000)
  • Remove installing qemu-binfmt from agent pipeline #2236 (@triarius)

v3.50.2

9 months ago

v3.50.2 (2023-07-21)

Full Changelog

This release contains a known issue:

Severity Description Fixed in
Medium When uploading pipelines, if any object in the pipeline YAML contained multiple merge keys, the pipeline would fail to parse. See below for a workaround ✅ Fixed in v3.50.3
Workaround for yaml merge key issue For example, this pipeline would fail to parse:
default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: *default_plugins
    <<: *default_retry
    command: "echo 'hello, world!'"

As a workaround for this, you can use yaml array merge syntax instead:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: [*default_plugins, *default_retry]
    command: "echo 'hello, world!'"

Fixed

  • Fix an issue introduced in #2207 where jobs wouldn't check if they'd been cancelled #2231 (@triarius)
  • Fix avoid-recursive-trap experiment not recognised #2235 (@triarius)
  • Further refactor to agent.JobRunner #2222 #2230 (@moskyb)

v3.50.1

9 months ago

v3.50.1 (2023-07-20)

Full Changelog

This release contains multiple issues:

Severity Description Fixed in
⚠️ Very High Jobs running on this version of the agent are not cancellable from the UI/API ✅ Fixed in v3.50.2
Medium When uploading pipelines, if any object in the pipeline YAML contained multiple merge keys, the pipeline would fail to parse. See below for a workaround ✅ Fixed in v3.50.3
Workaround for yaml merge key issue For example, this pipeline would fail to parse:
default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: *default_plugins
    <<: *default_retry
    command: "echo 'hello, world!'"

As a workaround for this, you can use yaml array merge syntax instead:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: [*default_plugins, *default_retry]
    command: "echo 'hello, world!'"

Fixed

  • Empty or zero-length steps is no longer a parser error, and is normalised to [] instead #2225, #2229 (@DrJosh9000)
  • Group steps now correctly include the group key #2226 (@DrJosh9000)
  • Increases to test coverage for the new parser #2227 (@DrJosh9000)

v3.50.0

9 months ago

v3.50.0 (2023-07-18)

Full Changelog

This release contains multiple issues:

Severity Description Fixed in
Medium When uploading pipelines, some group steps are not correctly parsed, and were ignored. ✅ Fixed in v3.50.1
Low Uploading pipelines with empty or zero-length steps failed, where they should've been a no-op. ✅ Fixed in v3.50.1
⚠️ Very High Jobs running on this version of the agent are not cancellable from the UI/API ✅ Fixed in v3.50.2
Medium When uploading pipelines, if any object in the pipeline YAML contained multiple merge keys, the pipeline would fail to parse. See below for a workaround ✅ Fixed in v3.50.3
Workaround for yaml merge key issue For example, this pipeline would fail to parse:
default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: *default_plugins
    <<: *default_retry
    command: "echo 'hello, world!'"

As a workaround for this, you can use yaml array merge syntax instead:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: [*default_plugins, *default_retry]
    command: "echo 'hello, world!'"

Added

  • We're working on making pipeline signing a feature of the agent! But it's definitely not ready for primetime yet... #2216, #2200, #2191, #2186, #2190, #2181, #2184, #2173, #2180 (@moskyb, @DrJosh9000)
  • Add option to configure location of Job Log tmp file #2174 (@yhartanto)
  • Add avoid-recursive-trap experiment to avoid a recursive trap #2209 (@triarius)
  • Load the AWS Shared Credentials for s3 operations #1730 (@lox)

Fixed

  • Add workaround for fatal: bad object errors when fetching from a git mirror #2218 (@DrJosh9000)
  • Fix missing fetch when updating git mirrors of submodules (https://github.com/buildkite/agent/pull/2203) (@DrJosh9000)
  • Use a unique name for each agent started using the systemd template unit file #2205 (@DavidGregory084)
  • Polyglot hooks wasn't documented in EXPERIMENTS.md, so we fixed that #2169 (@moskyb)
  • De-experimentify wording on the status page #2172 (@DrJosh9000)
  • The secrets redactor now properly redacts multi-line secrets and overlapping secrets #2154 (@DrJosh9000)

Changed

  • Print agent version and build in debug logs #2211 (@triarius)
  • Include the version each experiment was promoted #2199 (@DrJosh9000)

Various code cleanups and meta-fixes

  • Fix docker builds for Ubuntu 22.04 #2217 (@moskyb)
  • JobRunner cleanup #2207 (@moskyb)
  • Simplify command phase #2206 (@triarius)
  • Rename Bootstrap struct (and friends) to Executor #2188 (@moskyb)
  • Upgrade docker compose plugin to v4.14, use docker compose v2 #2189 (@moskyb)
  • Rename package bootstrap -> job #2187 (@moskyb)
  • Clarify code around creating a process group #2185 (@triarius)
  • Fix docker builds for Ubuntu 22.04 #2217 (@moskyb)

And the usual amount of @dependabot[bot] updates!