Celery Versions Save

Distributed Task Queue (development branch)

v5.4.0

1 week ago

Celery v5.4.0 and v5.3.x have consistently focused on enhancing the overall QA, both internally and externally. This effort led to the new pytest-celery v1.0.0 release, developed concurrently with v5.3.0 & v5.4.0.

This release introduces two significant QA enhancements:

  • Smoke Tests: A new layer of automatic tests has been added to Celery's standard CI. These tests are designed to handle production scenarios and complex conditions efficiently. While new contributions will not be halted due to the lack of smoke tests, we will request smoke tests for advanced changes where appropriate.
  • Standalone Bug Report Script: The new pytest-celery plugin now allows for encapsulating a complete Celery dockerized setup within a single pytest script. Incorporating these into new bug reports will enable us to reproduce reported bugs deterministically, potentially speeding up the resolution process.

Contrary to the positive developments above, there have been numerous reports about issues with the Redis broker malfunctioning upon restarts and disconnections. Our initial attempts to resolve this were not successful (#8796). With our enhanced QA capabilities, we are now prepared to address the core issue with Redis (as a broker) again.

The rest of the changes for this release are grouped below, with the changes from the latest release candidate listed at the end.

What's Changed

  • Add a Task class specialised for Django (#8491)
  • Add Google Cloud Storage (GCS) backend (#8868)
  • Added documentation to the smoke tests infra (#8970)
  • Added a checklist item for using pytest-celery in a bug report (#8971)
  • Bugfix: Missing id on chain (#8798)
  • Bugfix: Worker not consuming tasks after Redis broker restart (#8796)
  • Catch UnicodeDecodeError when opening corrupt beat-schedule.db (#8806)
  • chore(ci): Enhance CI with workflow_dispatch for targeted debugging and testing (#8826)
  • Doc: Enhance "Testing with Celery" section (#8955)
  • Docfix: pip install celery[sqs] -> pip install "celery[sqs]" (#8829)
  • Enable efficient chord when using dynamicdb as backend store (#8783)
  • feat(daemon): allows daemonization options to be fetched from app settings (#8553)
  • Fix DeprecationWarning: datetime.datetime.utcnow() (#8726)
  • Fix recursive result parents on group in middle of chain (#8903)
  • Fix typos and grammar (#8915)
  • Fixed version documentation tag from #8553 in configuration.rst (#8802)
  • Hotfix: Smoke tests didn't allow customizing the worker's command arguments, now it does (#8937)
  • Make custom remote control commands available in CLI (#8489)
  • Print safe_say() to stdout for non-error flows (#8919)
  • Support moto 5.0 (#8838)
  • Update contributing guide to use ssh upstream url (#8881)
  • Update optimizing.rst (#8945)
  • Updated concurrency docs page. (#8753)

Dependencies Updates

  • Bump actions/setup-python from 4 to 5 (#8701)
  • Bump codecov/codecov-action from 3 to 4 (#8831)
  • Bump isort from 5.12.0 to 5.13.2 (#8772)
  • Bump msgpack from 1.0.7 to 1.0.8 (#8885)
  • Bump mypy from 1.8.0 to 1.9.0 (#8898)
  • Bump pre-commit to 3.6.1 (#8839)
  • Bump pre-commit/action from 3.0.0 to 3.0.1 (#8835)
  • Bump pytest from 8.0.2 to 8.1.1 (#8901)
  • Bump pytest-celery to v1.0.0 (#8962)
  • Bump pytest-cov to 5.0.0 (#8924)
  • Bump pytest-order from 1.2.0 to 1.2.1 (#8941)
  • Bump pytest-subtests from 0.11.0 to 0.12.1 (#8896)
  • Bump pytest-timeout from 2.2.0 to 2.3.1 (#8894)
  • Bump python-memcached from 1.59 to 1.61 (#8776)
  • Bump sphinx-click from 4.4.0 to 5.1.0 (#8774)
  • Update cryptography to 42.0.5 (#8869)
  • Update elastic-transport requirement from <=8.12.0 to <=8.13.0 (#8933)
  • Update elasticsearch requirement from <=8.12.1 to <=8.13.0 (#8934)
  • Upgraded Sphinx from v5.3.0 to v7.x.x (#8803)

Changes since 5.4.0rc2

New Contributors

Full Changelog: https://github.com/celery/celery/compare/v5.3.6...v5.4.0

v5.4.0rc2

4 weeks ago

What's Changed

New Contributors

Full Changelog: https://github.com/celery/celery/compare/v5.4.0rc1...v5.4.0rc2

v5.4.0rc1

3 months ago

Celery v5.4 continues our effort to provide improved stability in production environments. The release candidate version is available for testing. The official release is planned for March-April 2024.

Please share feedback in the discussion below.

What's Changed

New Contributors

Full Changelog: https://github.com/celery/celery/compare/v5.3.6...v5.4.0rc1

v5.3.6

5 months ago

This release is focused mainly to fix AWS SQS new feature compatibility issue and old regressions. The code changes are mostly fix for regressions. More details can be found below.

What's Changed

New Contributors

Full Changelog: https://github.com/celery/celery/compare/v5.3.5...v5.3.6

v5.3.5

5 months ago

Main theme of this release is adding Python 3.12 compatibility support all through the projects dependencies. Also lots of bugs were squashed. Dependencies upgraded and docs improved.

What's Changed

New Contributors

Full Changelog: https://github.com/celery/celery/compare/v5.3.4...v5.3.5

v5.3.4

7 months ago

Notice Regarding v5.3.2 and v5.3.3

Versions 5.3.2 and 5.3.3 were identified to contain breaking changes and have been yanked to maintain codebase stability. We appreciate the community's timely feedback on these versions and apologize for any inconvenience caused.

What's Changed

New Contributors

Full Changelog: https://github.com/celery/celery/compare/v5.3.1...v5.3.4

v5.3.1

10 months ago

Release date: 2023-06-18 8:15 P.M GMT+6

Release by: Asif Saif Uddin

  • Upgrade to latest pycurl release (#7069).
  • Limit librabbitmq>=2.0.0; python_version < '3.11' (#8302).
  • Added initial support for python 3.11 (#8304).
  • ChainMap observers fix (#8305).
  • Revert optimization CLI flag behaviour back to original.
  • Restrict redis 4.5.5 as it has severe bugs (#8317).
  • Tested pypy 3.10 version in CI (#8320).
  • Bump new version of kombu to 5.3.1 (#8323).
  • Fixed a small float value of retry_backoff (#8295).
  • Limit pyro4 up to python 3.10 only as it is (#8324).

v5.3.0

10 months ago

Release date: 2023-06-06 12:00 P.M GMT+6

Release by: Asif Saif Uddin

  • Test kombu 5.3.0 & minor doc update (#8294).
  • Update librabbitmq.txt > 2.0.0 (#8292).
  • Upgrade syntax to py3.8 (#8281).

v5.3.0rc2

10 months ago

Release date: 2023-05-31 9:00 P.M GMT+6

Release by: Asif Saif Uddin

  • Add missing dependency.
  • Fix exc_type being the exception instance rather.
  • Fixed revoking tasks by stamped headers (#8269).
  • Support sqlalchemy 2.0 in tests (#8271).
  • Fix docker (#8275).
  • Update redis.txt to 4.5 (#8278).
  • Update kombu>=5.3.0rc2.

v5.3.0rc1

11 months ago

Release date: 2023-05-11 4:24 P.M GMT+2

Release by: Tomer Nosrati

  • fix functiom name by @cuishuang in #8087
  • Update CELERY_TASK_EAGER setting in user guide by @thebalaa in #8085
  • Stamping documentation fixes & cleanups by @Nusnus in #8092
  • switch to maintained pyro5 by @auvipy in #8093
  • udate dependencies of tests by @auvipy in #8095
  • cryptography==39.0.1 by @auvipy in #8096
  • Annotate celery/security/certificate.py by @Kludex in #7398
  • Deprecate parse_iso8601 in favor of fromisoformat by @stumpylog in #8098
  • pytest==7.2.2 by @auvipy in #8106
  • Type annotations for celery/utils/text.py by @max-muoto in #8107
  • Update web framework URLs by @sblondon in #8112
  • Fix contribution URL by @sblondon in #8111
  • Trying to clarify CERT_REQUIRED by @pamelafox in #8113
  • Fix potential AttributeError on 'stamps' by @Darkheir in #8115
  • Type annotations for celery/apps/beat.py by @max-muoto in #8108
  • Fixed bug where retrying a task loses its stamps by @Nusnus in #8120
  • Type hints for celery/schedules.py by @max-muoto in #8114
  • Reference Gopher Celery in README by @marselester in #8131
  • Update sqlalchemy.txt by @auvipy in #8136
  • azure-storage-blob 12.15.0 by @auvipy in #8137
  • test kombu 5.3.0b3 by @auvipy in #8138
  • fix: add expire string parse. by @Bidaya0 in #8134
  • Fix worker crash on un-pickleable exceptions by @youtux in #8133
  • CLI help output: avoid text rewrapping by click by @woutdenolf in #8152
  • Warn when an unnamed periodic task override another one. by @iurisilvio in #8143
  • Fix Task.handle_ignore not wrapping exceptions properly by @youtux in #8149
  • Hotfix for (#8120) - Stamping bug with retry by @Nusnus in #8158
  • Fix integration test by @youtux in #8156
  • Fixed bug in revoke_by_stamped_headers where impl did not match doc by @Nusnus in #8162
  • Align revoke and revoke_by_stamped_headers return values (terminate=True) by @Nusnus in #8163
  • Update & simplify GHA pip caching by @stumpylog in #8164
  • Update auth.txt by @auvipy in #8167
  • Update test.txt versions by @auvipy in #8173
  • remove extra = from test.txt by @auvipy in #8179
  • Update sqs.txt kombu[sqs]>=5.3.0b3 by @auvipy in #8174
  • Added signal triggered before fork by @jaroslawporada in #8177
  • Update documentation on SQLAlchemy by @max-muoto in #8188
  • Deprecate pytz and use zoneinfo by @max-muoto in #8159
  • Update dev.txt by @auvipy in #8192
  • Update test.txt by @auvipy in #8193
  • Update test-integration.txt by @auvipy in #8194
  • Update zstd.txt by @auvipy in #8195
  • Update s3.txt by @auvipy in #8196
  • Update msgpack.txt by @auvipy in #8199
  • Update solar.txt by @auvipy in #8198
  • Add Semgrep to CI by @Nusnus in #8201
  • Added semgrep to README.rst by @Nusnus in #8202
  • Update django.txt by @auvipy in #8197
  • Update redis.txt 4.3.6 by @auvipy in #8161
  • start removing codecov from pypi by @auvipy in #8206
  • Update test.txt dependencies by @auvipy in #8205
  • Improved doc for: worker_deduplicate_successful_tasks by @Nusnus in #8209
  • Renamed revoked_headers to revoked_stamps by @Nusnus in #8210
  • Ensure argument for map is JSON serializable by @candleindark in #8229