Prefect Versions Save

Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines

2.18.1

2 weeks ago

Release 2.18.1

Fixes

Experimental

Events and Automations

Engine

Documentation

Prefect UI Library

Integrations

Prefect CGP

New Contributors

2.18.0

3 weeks ago

Breaking Changes

Enhancements

Full Changelog: https://github.com/PrefectHQ/prefect/compare/2.17.0...2.18.0

See the release notes for more!

2.17.1

1 month ago

Fixes

All changes: https://github.com/PrefectHQ/prefect/compare/2.17.0...2.17.1

2.17.0

1 month ago

🧮 Manage Prefect variables via the Python SDK

Prefect variables are useful for storing and reusing data and configuration between and across workflows; and previously you could only create and update variables via the Prefect UI. With this release, you can now get and set Prefect variables directly in your Python code with the new Variable.set and Variable.get methods!

For an example of reading and writing variable values in Python see the following example:

from prefect.variables import Variable

# set a variable
variable = Variable.set(name="the_answer", value="42")

# get a variable
answer = Variable.get('the_answer')
print(answer.value)
# 42

# get a variable with a default value
answer = Variable.get('not_the_answer', default='42')
print(answer.value)
# 42

# update a variable
answer = Variable.set(name="the_answer", value="43", overwrite=True)
print(answer.value)
#43

Refer to the docs for more information and see the PR for implementation details: https://github.com/PrefectHQ/prefect/pull/12596

Other Enhancements 🌟

... and numerous 🐛 fixes!

Full Changelog: https://github.com/PrefectHQ/prefect/compare/2.16.9...2.17.0

See the release notes for more!

2.16.9

1 month ago

This release includes a number of enhancements and in-flight feature work.

🛠✨ One such enhancement helps streamline our CLI by adding a -jv/--job-variable option to prefect deploy, on par with the option available in prefect deployment run.

🔄🔗 In terms of enhancing existing Prefect concepts, we've removed a constraint that prevented tasks from being called from other tasks. For example, this allows you to call tasks within tasks within a flow.

🗿 📉 We no longer create artifacts for unpersisted results, which should prevent an influx of entries to the artifact table. Retried flows without persisted results will now have an error message stating that the "State data is missing" rather than referencing an "unpersisted result".

See the release notes for more details!

2.16.8

1 month ago

Fixes

2.16.7

1 month ago

💻 Introducing prefect shell for observing CLI commands

You can now observe CLI commands as a Prefect flow. For example:

» prefect shell watch "curl http://wttr.in/Chicago?format=3"
17:32:39.562 | INFO | prefect.engine - Created flow run 'powerful-mushroom' for flow 'Shell Command'
17:32:40.171 | INFO | Flow run 'powerful-mushroom' - Chicago: ⛅️ +50°F
17:32:40.315 | INFO | Flow run 'powerful-mushroom' - Finished in state Completed()

See these docs to learn how to:

  • run a shell command as a Prefect flow on-demand with watch
  • schedule a shell command as a recurring Prefect flow using serve

See the PR for implementation details: https://github.com/PrefectHQ/prefect/pull/11998

Other Enhancements 🌟

... and numerous 🐛 fixes!

Full Changelog: https://github.com/PrefectHQ/prefect/compare/2.16.6...2.16.7

See the release notes for more!

2.16.6

1 month ago

Fix new behavior in typer 0.10.0 that broke the prefect CLI

See the following pull request for implementation details:

All changes: https://github.com/PrefectHQ/prefect/compare/2.16.5...2.16.6

2.16.5

1 month ago

Multi-select deletion of flow runs

It is now easier to bulk select and delete flow runs through the UI. Listings of filterable and selectable flow runs (e.g. on the flow runs, flow, and deployment pages) now include a top-level checkbox for (de)selecting all currently filtered flow runs for bulk deletion.

image

See the following pull request for implementation details:

Visualize state changes and artifacts in the UI

Additionally, the flow run graph UI enhancements for visualizing state changes and artifacts added in 2.16.4 are now enabled by default. See the release notes in 2.16.4 for more details!

Full Changelog: https://github.com/PrefectHQ/prefect/compare/2.16.4...2.16.5

See the release notes for more!

2.16.4

1 month ago

Release 2.16.4

Flow Run Graph updates

The Flow Run Graph has been updated to display additional layers of information! Interactive and real-time state changes and artifacts are now visible in context on the graph.

The Prefect flow run graph

These new layers are available for opt-in usage via the PREFECT_EXPERIMENTAL_ENABLE_ARTIFACTS_ON_FLOW_RUN_GRAPH and PREFECT_EXPERIMENTAL_ENABLE_STATES_ON_FLOW_RUN_GRAPH settings.

Agents

A year ago, we released workers as a replacement for agents. Workers significantly enhance the experience of deploying flows and simplify the specification of each flow's infrastructure and runtime environment.

With this release we are adding a six month (September 14) deprecation warning to agents and related concepts. Please note that:

  • Deprecation will not impact or break any work running with agents and agent-related concepts - although we encourage users to upgrade to workers because they provide a better deployment experience, you can continue to use existing agents and related concepts after deprecation
  • After September 14, Prefect Cloud users will not be able to create new agent work pools or infrastructure blocks
  • After September 14, new minor versions of the Prefect Python package will not include agents

Like agents, workers support creating deployments through the Prefect CLI and through Python, but require different syntax. For more information, please refer to the Upgrade from Agents to Workers guide.

Enhancements

Fixes

Experimental

Deprecations

Typing

Documentation

New Contributors

Full Changelog: https://github.com/PrefectHQ/prefect/compare/2.16.3...2.16.4