Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Full Changelog: https://github.com/spotify/luigi/compare/3.4.0...3.5.0
Full Changelog: https://github.com/spotify/luigi/compare/3.3.0...3.4.0
@requires
#3077traceback_max_length
parameter for error email notifications #3086Unfulfilled dependency
error #3105OptionalParameter
datatype options #3079batch_email.email_interval
to be set in config as email-internal-minutes
#3125TimeDeltaParameter
to accept input as "seconds" #3125EnumListParameter
to top-level attribute imports #3144metrics_custom_import
for MetricCollectors
#3146configure_job
BigQuery property #3098run_task_kwargs
property to ECS #3083pickle_protocol
attribute and configuration option to Spark #3001pg8000
driver support to Postgres #3142task.disable_window
#3081LocalTarget
when is_tmp
attribute dne #3085luigi.format.Nop
import #3047cls.get_task_namespace()
#3129notifications_test.py
#3151max_shown_tasks
and max_graph_nodes
documentation to correct section #3156Task
's Register
metaclass #3154rdbms.CopyToTable
's _add_metadata_columns()
#3011@Tarrasch
from codeowners #3127tenacity
version #3147OptionalParameter
#3130prometheus-client
package #3163