Buildbot Versions Save

Bug fixes

• GitPoller now ensures the SSH Private Key it uses has a trailing newline.
• Migrated off python-future which prevented installing Buildbot on distributions that do not provide that package.
• Fixed sporadic navigation to builders page when new build is started (#7307).

Bug fixes

• Declare Python 3.12 compatibility in generated packages of master and worker

Features

• Added a new WSGI dashboards plugin for React frontend. It is backwards compatible with AngularJS one but may require changes in CSS styling of displayed web pages.
• Implemented a report generator (BuildSetCombinedStatusGenerator) that can access complete information about a buildset.
• Low level database API now has get_sourcestamps_for_buildset to get source stamps for a buildset. "/buildsets/:buildsetid/sourcestamps" endpoint has been added to access this from the Data API.
• Added buildset information to dictionaries returned by report generators.
• Added a way to pass additional reporter-specific data to Reporters. Added extra_info_cb argument to MessageFormatter for this use case.
• Implemented support for report generators in GerritStatusPush.

Deprecations and Removals

• The reviewCB, reviewArg, startCB, startArg, summaryCB, summaryArg, builders , wantSteps, wantLogs arguments of GerritStatusPush have been deprecated.

Bug fixes

• Fixed support for Twisted 23.10 and Python 3.12.
• Fixed Data API to have "parent_buildid" key-value pair in messages for rebuilt buildsets (#7222).
• Improved security of tarfile extraction to help avoid CVE-2007-4559. See more details in https://peps.python.org/pep-0706/. Buildbot uses filter='data' now. (#7294)
• Fixed web frontend package build on certain Python versions (e.g. 3.9).

Bug fixes

• buildbot.changes.bitbucket.BitbucketPullrequestPoller has been updated to emit the change files.
• Fixed build status key sent to Bitbucket exceeding length limits (#7249).
• Fixed a race condition resulting in EXCEPTION build results when build steps that are about to end are cancelled.
• Buildrequests are now selected by priority and then by buildrequestid (previously, Buildbot used the age as the secondary sort parameter). This preserves the property of choosing the oldest buildrequest, but makes it predictable which buildrequest will be selected, as there might be multiple buildrequests with the same age.
• Fixed worker to fail a step uploadDirectory instead of throwing an exception when directory is not available. (#5878)
• Added missing parent_buildid and parent_relationship keys to the buildset completion event in the Data API.
• Improved handling of Docker containers that fail before worker attaches to master. In such case build will be restarted immediately instead of waiting for a timeout to expire.
• The accessibility of secret files has been enhanced by enabling group-readability. Previously, secret files were exclusively accessible to the owner. Now, accessibility has been expanded to allow group members access as well. This enhancement is particularly beneficial when utilizing Systemd's LoadCredential feature, which configures secrets with group-readable (0o440) permissions.
• MailNotifier now works correctly when SSL packages are installed but useTls=False and auth (smtpUser, smtpPassword) is not set. (#5609)
• P4 now reports the correct got_revision when syncing a changelist that only delete files
• P4 step now use the rev-spec format //{p4client}/...@{revision} when syncing with a revision
• Fixed incorrect propagation of option --proxy-connection-string into buildbot.tac when creating new Worker.
• Fixed link to Builder in React Grid View.
• Addressed a number of timing errors in Nightly scheduler by upgrading croniter code.

Changes

• Buildbot will render step properties and check if step should be skipped before acquiring locks. This allows to skip waiting for locks in case step is skipped.
• The isRaw and isCollection attributes of the Endpoint type have been deprecated. Endpoint is used to extend the Buildbot API. As a replacement use the new kind attribute.
• AbstractLatentWorker.check_instance() now accepts error message being supplied in case instance is not good. The previous API has been deprecated.
• The published Docker image for the worker now uses Debian 11 (Bullseye) as base image.
• The published Docker image for the worker now runs Buildbot in virtualenv.

Improved Documentation

• Describe an existing bug with Libvirt latent workers that does not use a copy of the image (#7122).

Features

• The new React-based web frontend is no longer experimental. To enable please see the documentation on upgrading to 4.0 for more information. The new web frontend includes the following improvements compared to legacy AngularJS web frontend:

  • Project support (initially released in Buildbot 3.9.0).
  • Steps now show the amount of time spent waiting for locks.
  • The log viewer now supports huge logs without problems.
  • The log viewer now includes a search box that downloads entire log on-demand without additional button click.
  • The log viewer now supports downloading log file both as a file and also showing it inline in the browser.
  • The colors of the website can be adjusted from Buildbot configuration via www["theme"] key.
  • Buildsteps and pending buildrequests have anchor links which allows linking directly to them from external web pages.

• Workers can now be created to use connection string right out of the box when new option --connection-string= is used.

• Docker Latent workers will now show last logs in Buildbot UI when their startup fails.

• Added EndpointKind.RAW_INLINE data API endpoint type which will show the response data inline in the browser instead of downloading as a file.

• Implemented a way to specify volumes for containers spawned by KubeLatentWorker.

• Nightly scheduler now supports forcing builds at specific times even if onlyIfChanged parameter was true and there were no important changes.

• buildbot.steps.source.p4.P4 can now take a p4client_type argument to set the client type (More information on client type here)

• Added data and REST APIs to retrieve only projects with active builders.

• Improved step result reporting to specify whether step failed due to a time out.

• Added tags option to the Git source step to download tags when updating repository.

• Worker now sends failure_reason update when the command it was running timed out.

Deprecations and Removals

• Legacy AngularJS web frontend will be removed in Buildbot 4.0. Fixes to React web frontend that are regressions from AngularJS web frontend will be backported to 3.x Buildbot series to make migration easier.

• Buildbot Master now requires Python 3.8 or newer. Python 3.7 is no longer supported.

• buildbot.util.croniter module has been deprecated in favor of using Pypi croniter package.

• master.data.updates.setWorkerState() has been deprecated. Use master.data.updates.set_worker_paused() and master.data.updates.set_worker_graceful() as replacements.

• Buildbot now requires docker of version v4.0.0 or newer for Docker support.

• BuildStep instances are now more strict about when their attributes can be changed. Changing attributes of BuildStep instances that are not yet part of any build is most likely an error. This is because such instances are only being used to configure a builder as a source to create real steps from. In this scenario any attribute changes are ignored as far as build configuration is concerned.

  Such changing of attributes has been deprecated and will become an error in the future release.

  For customizing BuildStep after an instance has already been created set_step_arg(name, value) function has been added.

Bug fixes

• Work around requirements parsing error for the Twisted dependency by pinning Twisted to 22.10 or older. This fixes buildbot crash on startup when newest Twisted is installed.

Bug fixes

• Fixed handling of primary key columns on Postgres in the copy-db script.
• Fixed a race condition in the copy-db script which sometimes lead to no data being copied.
• Options for create-worker that are converted to numbers are now also checked to be valid Python literals. This will prevent creating invalid worker configurations, e.g.: when using option --umask=022 instead of --umask=0o022 or --umask=18 (#7047)
• Fixed worker not connecting error when there are files in WORKER/info folder that can not be decoded. (#3585, #4758, #6932)
• Fixed incorrect git command line parameters when using Git source step with mode="incremental", shallow=True, submodules=True (regression since Buildbot 3.9.0) (#7054).

Improved Documentation

• Clarified that shallow option for the Git source step is also supported in incremental mode.

Bug fixes

• Fixed missed invocations of methods decorated with util.debounce when debouncer was being stopped under certain conditions. This caused step and build state string updates to be sometimes missed.
• Improved stale connection handling in GerritChangeSource. GerritChangeSource will instruct the ssh client to send periodic keepalive messages and will reconnect if the server does not reply for 45 seconds (default). GerritChangeSource now has ssh_server_alive_interval_s and ssh_server_alive_count_max options to control this behavior.
• Fixed unnecessary build started under the following conditions: there is an existing Nightly scheduler, onlyIfChanged is set to true and there is version upgrade from v3.4.0 (#6793).
• Fixed performance of changes data API queries with custom filters.
• Prevent possible event loss during reconfig of reporters (#6982).
• Fixed exception thrown when worker copies directories in Solaris operating system (#6870).
• Fixed excessive log messages due to JWT token decoding error (#6872).
• Fixed excessive log messages when otherwise unsupported /auth/login endpoint is accessed when using RemoteUserAuth authentication plugin.

Features

• Introduce a way to group builders by project. A new projects list is added to the configuration dictionary. Builders can be associated to the entries in that list by the new project argument.

  Grouping builders by project allows to significantly clean up the UI in larger Buildbot installations that contain hundreds or thousands of builders for a smaller number of unrelated codebases. This is currently implemented only in experimental React UI.

• Added support specifying the project in GitHubPullrequestPoller. Previously it was forced to be equal to GitHub's repository full name.

• Reporter BitbucketServerCoreAPIStatusPush now supports BuildRequestGenerator and generates build status for build requests (by default).

• Buildbot now has copy-db script migrate all data stored in the database from one database to another. This may be used to change database engine types. For example a sqlite database may be migrated to Postgres or MySQL when the load and data size grows.

• Added cron features like last day of month to Nightly Scheduler.

• Buildrequests can now have their priority changed, using the /buildrequests API.

• The force scheduler can now set a build request priority.

• Added support for specifying builder descriptions in markdown which is later rendered to HTML for presentation in the web frontend.

• Build requests are now sorted according to their buildrequest. Request time is now used as a secondary sort key.

• Significantly improved performance of reporters or reporters with slower generators which is important on larger Buildbot installations.

• Schedulers can now set a default priority for the buildrequests that it creates. It can either be an integer or a function.

• Implement support for shallow submodule update using git.

• GerritChangeSource will now log a small number of previous log lines coming from ssh process in case of connection failure.

Deprecations and Removals

• Deprecated projectName and projectURL configuration dictionary keys.

Bug fixes

• Fixed compatibility issues with Python 3.11.
• Fixed compatibility with Autobahn v22.4.1 and newer.
• Fixed issue with overriding env when calling ShellMixin.makeRemoteShellCommand
• Buildbot will now include the previous location of moved files when evaluating a Github commit. This fixes an issue where a commit that moves a file out of a folder, would not be shown in the web UI for a builder that is tracking that same folder.
• Improved reliability of Buildbot log watching to follow log files even after rotation. This improves reliability of Buildbot start and restart scripts.
• Fixed handling of occasional errors that happen when attempting to kill a master-side process that has already exited.
• Fixed a race condition in PyLint step that may lead to step throwing exceptions.
• Fixed compatibility with qemu 6.1 and newer when using LibVirtWorker with cheap_copy=True (default).
• Fixed an issue with secrets provider stripping newline from ssh keys sent in git steps.
• Fixed occasional errors that happen when killing processes on Windows. TASKKILL command may return code 255 when process has already exited.
• Fixed deleting secrets from worker that contain '~' in their destination path.

Changes

• Buildbot now requires NodeJS 14.18 or newer to build the frontend.
• The URLs emitted by the Buildbot APIs have been changed to include slash after the hash (#) symbol to be compatible with what React web UI supports.

Improved Documentation

• Replace statement "https is unsupported" with a more detailed disclaimer.

Features

• Add a way to disable default WarningCountingShellCommand parser.
• Added health check API that latent workers can use to specify that a particular worker will not connect and build should not wait for it and mark itself as failure immediately.
• Implemented a way to customize TLS setting for LdapUserInfo.

Bug fixes

• Improved statistics capture to avoid negative build duration.
• Improved reliability of "buildbot stop" (#3535).
• Cancelled builds now have stop reason included into the state string.
• Fixed custom_class change hook checks to allow hook without a plugin.
• Added treq response wrapper to fix issue with missing url attribute.
• Fixed Buildbot Worker being unable to start on Python 2.7 due to issue in a new version of Automat dependency.

Features

• Expanded ChangeFilter filtering capabilities:
  • New <attribute>_not_eq parameters to require no match
  • <attribute>_re now support multiple regexes
  • New <attribute>_not_re parameters to require no match by regex
  • New property_<match_type> parameters to perform filtering on change properties.
• Exposed frontend configuration as implementation-defined JSON document that can be queried separately.
• Added support for custom branch keys to OldBuildCanceller. This is useful in Version Control Systems such as Gerrit that have multiple branch names for the same logical branch that should be tracked by the canceller.
• p4port argument of the P4 step has been marked renderable.
• Added automatic generation of commands for Telegram bot without need to send them manually to BotFather.

Deprecations and Removals

• This release includes an experimental web UI written using React framework. The existing web UI is written using AngularJS framework which is no longer maintained. The new web UI can be tested by installing buildbot-www-react package and 'base_react': {} key-value to www plugins. Currently no web UI plugins are supported. The existing web UI will be deprecated on a subsequent Buildbot release and eventually replaced with the React-based web UI on Buildbot 4.0.

Bug fixes

• Fixed handling of last line in logs when Buildbot worker 3.5 and older connects to Buildbot master 3.6 using PB protocol (#6632).
• Fixed worker cpdir command handling when using PB protocol (#6539)