Mara Pipelines Versions Save

• add entry point mara.commands (for mara-cli support)
• add mara-pipelines click command group (#104)
• add dynamic Task (#106)
• add max_retries for parallel tasks (#105)
• fix html_doc_items tuple of command ReadScriptOutput

• upgrade to bootstrap 4.6 (from alpha 4) (#66)
  Caution: Some CSS classes changed between Boostram Alpha 4 and 4.6. You might need to upgrade other mara packages as well, e.g. mara-pipelines and mara-app.
• add HttpRequest command #78 (#79)
• add WriteFile command (#89)
• add support for formats in file operations (#95)
• add typing (#91)
• add before/after task to ParallelTask only when not command list is not empty (#93)
• fix get_user_display_name on docker (#90)
• fix small issues (#91)
• fix SQLAlchemy warning about declarative_base moved in 2.0 (#99)

required changes You might need to investigate your custom CSS styling, see boostram upgrade above.

• Fix CopyIncrementally with no data (#54)
• Add ability to specify modification value type in CopyIncrementally (#53) 66e7dc1 Jan Katins [email protected] 4. Mar 2021 at 22:06
• Fix read stderr during command execution (#47)
• Use echo_queries from mara_db.config.default_echo_queries (#58)
• Include all versioned package files in wheel

• Add option to hide system stats in the UI (#72)
• Add option to disable the collection of system statistics (#72)
• Add syntax highlighting for TSQL and SQLite3 (#86)
• Support pipeline execution without 'mara' database (#71)
• Fix getting exitcode from process issue since python 3.8 (#87)
• Use client-side rendering for graphviz when shell command is not available (#70)

• Fix for visible passwords in the logs despite mara_pipelines.config.password_masks() set. Bug was introduced in 3.0.0.

• Ignore not succeeded executions in cost calculation (#36)
• Ensure we log errors via events in case of error/shutdown (#33)
• Fix a bug where we reported the wrong error to chat channels when running in the browser and did not restart between failed runs (#33)

• Fix Problems when frontend and database are in a different timezone (#34)

• Make event handlers configurable: this allows for e.g. adding your own notifier for specific events
• Switch slack to use events for notifications of interactive pipeline runs
• Fix an edge case bug where reverting a commit after an error in the table creation for an incremental load job would not recreate the original tables leading to a failed load
• Fix an edge case bug where crashing during a triggered (code change, TRUNCATE) full load of an incremental load job after the table was already loaded would not rerun the full load leading to missing data
• Optimize how we set the spawning method in multiprocessing

Including the file_dependencies as variable for the copy command could handle cases in ETL pipeline, where the copy command shall be skipped if the sql_files stay the same.