Trains Versions Save

Bug Fixes

• Fix auto-scaler should recheck that the worker is still IDLE before shutting it down (#1240, thanks @cthorey!)
• Fix resource monitor fails to get GPU stats in some edge cases

New Features

• Add draft option to pipeline steps (#1226, thanks @CharlesFrankum!)
• Add support for custom working directory for pipelines (#1194)
• Add Task.get_requirements() method returning the task’s requirements
• Allow controlling the number of threads used by StorageManager.download_folder() using the max_workers argument
• Update examples dependencies
• Improve auto populate in Task.init()
• Documentation:
  • Add docstrings for model properties
  • Improve docstring for force_requirements_env_freeze
  • Add Task.set_packages() docstring notes

Bug Fixes

• Fix UTF-8 script code encoding issue (#1208, thanks @ae-ae!)
• Fix Colab docs (#1220, thanks @tkukurin!)
• Fix metrics reporting with OutputModel while in offline mode results in an error (#1172)
• Fix task running in Google Colab doesn't properly get the notebook diff (#1204)
• Fix hydra binds break in offline mode (#1215)
• Fix HPO crashes when optimizing for single value scalars (#1221)
• Fix GPU info such as gpu_memory and gpu_type is not being collected in some cases
• Fix clearml-data CLI tool will move non-dataset tasks to a .dataset project if the respective task is not a dataset
• Fix pandas DataFrame artifacts with hierarchical indices get mangled by the CSV round-trip
• Fix urllib3 sends deprecation warning when setting ssl_version
• Fix Task.connect(dict) return value is not dict-compatible
• Fix jsonargparse sub-command config parsing
• Fix Lightning integration crashes when a config entry contains . in its name
• Fix Python 3.5 compatibility

Bug Fixes

• Fix AWS S3 configuration secure property not forwarded in StorageHelper (#1205, thanks @odulcy-mindee!)
• Fix NVIDIA_VISIBLE_DEVICES=all env var value will result in no GPU metrics being reported
• Fix setting output_uri when an AWS S3 config section with secure=false exists raises an error
• Fix CacheManager attempts to clear cache when file limit has not been reached resulting in slow unnecessary checks

New Features and Bug Fixes

• Add hash check for pickle files (resolves CVE-2024-24590)
• Fix clearml-task relative paths calculation when --cwd, --folder and --script are provided (#1161, thanks @dimidagd!)
• Fix regression causing an error when secure is provided in AWS S3 bucket settings

New Features and Improvements

• Report upload/download with tqdm if installed
• Add support for a PipelineController.version property
• Add support for specifying an AWS profile when providing bucket storage credentials
• Add an ignore_remote_overrides argument to Task.connect() and Task.connect_configuration() allowing users to disable overriding values when running remotely
• Add support for setting a custom certificate path using the CLEARML_API_HOST_VERIFY_CERT environment variable (instead of just allowing the user to disable verification)
• Add dataset reporting example (#1180)
• Update README GIFs (#1191)

Bug Fixes

• Fix missing component callbacks on multiple step calls (#1195, thanks @materight!)
• Fix unsafe usage of tempfile.mktemp
• Fix potential path traversal on file download (CVE-2024-24591)
• Fix Python 3.12 support by not using distutils
• Fix TaskScheduler does not save local state before running remotely
• Fix pipeline should continue instead of aborting when PipelineController.abort_on_failure is set to False and a step has been aborted
• Fix HPO crashes when its corresponding task is initialized with continue_last_task=True

Bug Fixes

• Fix created pipelines do not appear in the UI pipelines section

New Features and Improvements

• Add support for decorated pipeline steps (#1154)
• Improve torch.distributed behavior and auto-detect Huggingface accelerate distributed execution
• Improve jsonargparse integration: allow_config_file_override_from_ui argument added for enabling/disabling overriding configuration file entries
• Add raise_on_error parameter to Model.update_weights() and Model.update_weights_package()
• Add HPO multi-objective optimization support by specifying multiple optimization objectives to the HyperParameterOptimizer class
• Improve pipelines from functions: libraries used by pipeline steps no longer have to be imported in the step itself and can be imported globally instead
• Support clearing repository details in task.set_repo()
• Auto detect tensorflow-intel as tensorflow
• Add support for auto detecting torch and transformers accelerate distributed execution
• Add tabular data multi-stage pipeline process example

Bug Fixes

• Fix langchain integration (#1124, #1126)
• Fix using target_project with TaskScheduler.add_task() corrupts project if used with pipelines (#1137)
• Fix PipelineController.add_function_step() fails when adding custom decorated step (#1154)
• Fix jsonargparse binding does not capture parameters before Task.init() is called (#1164)
• Fix invalid jsonargparse arguments should be deleted
• Fix handling of appended parameters to Hydra defaults list
• Fix Task.set_repo() and Task.set_packages() do not wait for automatic asynchronous repository detection
• Fix active GPU filtering in resource monitor to support GPU ID and not just GPU idx
• Fix misleading error raised when using the SDK with no configuration file or an invalid configuration file
• Fix process sometimes hangs issue by improving exit and exception handlers, unregistering signal handling in child processes except for the BackgroundMonitor

New Features

• Support parallel dataset verification (#1131, thanks @charlienewey-odin!)
• Support lightgbm model loading from a string (#1136, thanks @bedapisl!)
• Add support for Hydra command-line syntax for modifying omegaconf
• Add support for setting output_uri in pipeline decorators and functions
• Add more visibility when overriding jsonargparse arguments
• Use StorageManager cache when downloading models
• Support parsing GS credentials as a JSON string if they do not point to a JSON file
• Support ignoring default_output_uri on remote runs when Task.init(output_uri=False) was specified and the backend task data has no output_uri specified
• Add configuration files logging example
• Support raising LoginError when initializing
• Add HuggingFace Transformers Example (#1146)
• Improve docstrings for tag filters
• Add a security policy file (#1148)

Bug Fixes

• Fix Google Storage subdir path in bucket URL breaks validation (#1117, thanks @materight!)
• Fix jsonschema DeprecationWarning in newer versions (#1128, thanks @xquyvu!)
• Fix task.set_script(diff="") does not clear the uncommitted changes (#1106)
• Fix parallel dataset verification fails on older Python versions (#1144)
• Fix attrs.__version__ deprecation warning (#1145)
• Fix Task.get_parameters_as_dict(cast=True) raises a YAML parser error
• Fix potential threading issues when setting self.lock to None
• Fix pipeline version is not displayed in the table view and version list
• Fix displayed pipeline run time in pipeline components is not the same as that shown in the info tab in full detail
• Fix Model.connect() in remote execution might result in the wrong model being connected
• Fix exception raised on ConnectionError (which actually indicates that a read has timed out, and should be retried)
• Fix missing configuration error not raised in certain cases
• Fix loading queues containing metadata fails due to an incorrect queue metadata scheme
• Fix pyjwt requirements for Python < 2.9
• Fix pipeline examples

Bug Fixes

• Fix Hydra overrides integration (#1120)
• Fix GPU memory used reports 0 when memory can not be queried per process (edge case)

New Features

• Add support for recursive list, dict, and tuple ref parsing for pipeline controller.add step() parameter overrides (#1099, thanks @natephysics!)
• Update PyNVML to the latest NVIDIA version for better GPU monitoring
• Add force_single_script_file argument to Task.create() to avoid git repository auto detection
• Use os.register_at_fork instead of monkey patching fork for python > 3.6
• Add support for programmatically archive and unarchive models from the model registry using the Model.archive() and Mode.unarchive() methods (#1096)
• Improve documentation

Bug Fixes

• Fix Hydra overrides integration (#552)
• Fix model reporting in TensorFlow 2.13 does not work properly (#1112)
• Fix GPU memory used reports 0 when memory can not be queried per process
• Fix FastAI binding not reporting scalars if TensorBoard is installed
• Fix JSON serialization error does not raise exception
• Fix OutputModel should prefer connected task name over filename stem
• Fix dataset zip extraction might fail when creating folders concurrently (with Exception [Errno 17] File exists warning)
• Fix import sys should not appear in pipeline component auto detected packages
• Fix compatibility issue with requests < 2.28