:rocket: Build and manage real-life ML, AI, and data science projects with ease!
When running a task with the @retry decorator, previously we were able to only view the logs of the latest attempt of the task.
With this release it is now possible to target a specific attempt with the --attempt option for the logs command
python example.py logs 123/retry_step/456 --attempt 2
This release introduces a new command for scrubbing log contents of tasks in case they contain sensitive information that needs to be redacted.
Simplest use case is scrubbing the latest task logs. By default both stdout and stderr are scrubbed
python example.py logs scrub 123/example/456
There are also options to target only a specific log stream
python example.py logs scrub 123/example/456 --stderr
python example.py logs scrub 123/example/456 --stdout
when using the@retry decorator, tasks can have multiple attempts with separate logs that require scrubbing. By default only the latest attempt is scrubbed. There are options to make scrubbing multiple attempts easier
# scrub specific attempt
python example.py logs scrub 123/retry_step/456 --attempt 1
# scrub all attempts
python example.py logs scrub 123/retry_step/456 --all
# scrub specified attempt and all prior to it (this would scrub attempts 0,1,2,3)
python example.py logs scrub 123/retry_step/456 --all --attempt 3
The command also accepts only specifying a step for scrubbing. This is useful for steps with multiple tasks, like a foreach split.
python example.py logs scrub 123/foreach_step
all the above options also apply when targeting a step for scrubbing.
Note: Log scrubbing for running tasks is not recommended, and is actively protected against. There can be occasions where a task has failed in such a way that it still counts as not completed. For such a case you can supply the --include-not-done option to try and scrub it as well.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.14...2.11.15
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.13...2.11.14
This release introduces configuration options for setting default values for cpu / memory / disk when running on Kubernetes. These can be set either with environment variables
METAFLOW_KUBERNETES_CPU=
METAFLOW_KUBERNETES_MEMORY=
METAFLOW_KUBERNETES_DISK=
or in a Metaflow profile
{
"KUBERNETES_CPU": "",
"KUBERNETES_MEMORY": "",
"KUBERNETES_DISK": "",
}
These values will be overruled by specifying a value through the @kubernetes or @resources decorators.
This release changes the way task ids are generated on Argo Workflows in order to solve an issue where extremely wide foreach splits could not execute correctly due to hard limits on input parameters size on Argo Workflows.
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.12...2.11.13
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.11...2.11.12
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.10...2.11.11
This release fixes an issue where partial or empty argo event payloads would incorrectly overwrite the default values for the parameters of a triggered flow.
For example a flow with
@trigger(events=["params_event"])
class DefaultParamEventFlow(FlowSpec):
param_a = Parameter(
name="param_a",
default="default value A",
)
param_b = Parameter(
name="param_b",
default="default value B",
)
will now correctly have the default values for its parameters when triggered by
from metaflow.integrations import ArgoEvent
ArgoEvent('params_event').publish()
or a default value for param_b and the supplied value for param_a when triggered by
ArgoEvent('params_event').publish({"param_a": "custom-value"})
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.9...2.11.10
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.8...2.11.9
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.7...2.11.8
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.6...2.11.7
Full Changelog: https://github.com/Netflix/metaflow/compare/2.11.5...2.11.6