TFX is an end-to-end platform for deploying production ML pipelines
Package Name | Version Constraints | Previously (in v1.14.0 ) |
Comments |
---|---|---|---|
kfp-pipeline-spec |
kfp-pipeline-spec>=0.1.10,<0.2 |
>0.1.13,<0.2 |
ph.make_proto()
allows constructing proto-valued placeholders, e.g. for
larger config protos fed to a component.ph.join_path()
is like os.path.join()
but for placeholders.experimental_debug_stripper
into the Transform
pipeline runner.Placeholder
and all subclasses have been moved to other modules, their
structure has been changed and they're now immutable. Most users won't care
(the main public-facing API is unchanged and behaves the same way). If you
do special operations like isinstance()
or some kind of custom
serialization on placeholders, you will have to update your code.placeholder.Placeholder.traverse()
now returns more items than before,
namely also placeholder operators like _ConcatOperator
(which is the
implementation of Python's +
operator).placeholder.RuntimeInfoKey
enumeration was removed. Just hard-code the
appropriate string values in your code, and reference the new Literal
type
placeholder.RuntimeInfoKeys
if you want to ensure correctness.@component
must now be passed as kwargs and its return type
is changed from being a Type
to just being a callable that returns a new
instance (like the type's initializer). This will allow us to instead return
a factory function (which is not a Type
) in future. For a given
@component def C()
, this means:
C
as a type anymore. For instance, replace
isinstance(foo, C)
with something else. Depending on your use case, if
you just want to know whether it's a component, then use
isinstance(foo, tfx.types.BaseComponent)
or
isinstance(foo, tfx.types.BaseFunctionalComponent)
.
If you want to know which component it is, check its .id
instead.
Existing such checks will break type checking today and may additionally
break at runtime in future, if we migrate to a factory function.C.test_call()
like before, and it will
continue to be supported in future.foo: C
break and must be replaced with
foo: tfx.types.BaseComponent
or
foo: tfx.types.BaseFunctionalComponent
.C.EXECUTOR_SPEC
breaks
type checking today and should be migrated away from. In particular, for
.EXECUTOR_SPEC.executor_class().Do()
in unit tests, use .test_call()
instead.C
, this
can now lead to (justified) type checking errors that were previously
hidden due to C
being of type Any
.ph.to_list()
was renamed to ph.make_list()
for consistency.Package Name | Version Constraints | Previously (in v1.14.0 ) |
Comments |
---|---|---|---|
keras-tuner |
>=1.0.4,<2,!=1.4.0,!=1.4.1 |
>=1.0.4,<2 |
|
packaging |
>=20,<21 |
>=22 |
|
attrs |
19.3.0,<22 |
19.3.0,<24 |
|
google-cloud-bigquery |
>=2.26.0,<3 |
>=3,<4 |
|
tensorflow |
>=2.15,<2.16 |
>=2.13,<2.14 |
|
tensorflow-decision-forests |
>=1.0.1,<1.9 |
>=1.0.1,<2 |
|
tensorflow-hub |
>=0.9.0,<0.14 |
>=0.15.0,<0.16 |
|
tensorflow-serving |
>=1.15,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,<3 |
>=2.15,<2.16 |
ph.make_proto()
allows constructing proto-valued placeholders, e.g. for
larger config protos fed to a component.ph.join_path()
is like os.path.join()
but for placeholders.experimental_debug_stripper
into the Transform
pipeline runner.Placeholder
and all subclasses have been moved to other modules, their
structure has been changed and they're now immutable. Most users won't care
(the main public-facing API is unchanged and behaves the same way). If you
do special operations like isinstance()
or some kind of custom
serialization on placeholders, you will have to update your code.placeholder.Placeholder.traverse()
now returns more items than before,
namely also placeholder operators like _ConcatOperator
(which is the
implementation of Python's +
operator).placeholder.RuntimeInfoKey
enumeration was removed. Just hard-code the
appropriate string values in your code, and reference the new Literal
type
placeholder.RuntimeInfoKeys
if you want to ensure correctness.@component
must now be passed as kwargs and its return type
is changed from being a Type
to just being a callable that returns a new
instance (like the type's initializer). This will allow us to instead return
a factory function (which is not a Type
) in future. For a given
@component def C()
, this means:
C
as a type anymore. For instance, replace
isinstance(foo, C)
with something else. Depending on your use case, if
you just want to know whether it's a component, then use
isinstance(foo, tfx.types.BaseComponent)
or
isinstance(foo, tfx.types.BaseFunctionalComponent)
.
If you want to know which component it is, check its .id
instead.
Existing such checks will break type checking today and may additionally
break at runtime in future, if we migrate to a factory function.C.test_call()
like before, and it will
continue to be supported in future.foo: C
break and must be replaced with
foo: tfx.types.BaseComponent
or
foo: tfx.types.BaseFunctionalComponent
.C.EXECUTOR_SPEC
breaks
type checking today and should be migrated away from. In particular, for
.EXECUTOR_SPEC.executor_class().Do()
in unit tests, use .test_call()
instead.C
, this
can now lead to (justified) type checking errors that were previously
hidden due to C
being of type Any
.ph.to_list()
was renamed to ph.make_list()
for consistency.Package Name | Version Constraints | Previously (in v1.14.0 ) |
Comments |
---|---|---|---|
keras-tuner |
>=1.0.4,<2,!=1.4.0,!=1.4.1 |
>=1.0.4,<2 |
|
packaging |
>=20,<21 |
>=22 |
|
attrs |
19.3.0,<22 |
19.3.0,<24 |
|
google-cloud-bigquery |
>=2.26.0,<3 |
>=3,<4 |
|
tensorflow |
>=2.15,<2.16 |
>=2.13,<2.14 |
|
tensorflow-decision-forests |
>=1.0.1,<1.9 |
>=1.0.1,<2 |
|
tensorflow-hub |
>=0.9.0,<0.14 |
>=0.15.0,<0.16 |
|
tensorflow-serving |
>=1.15,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,<3 |
>=2.15,<2.16 |
Placeholder
(and _PlaceholderOperator
) are no longer Jsonable
.Package Name | Version Constraints | Previously (in v1.13.0 ) |
Comments |
---|---|---|---|
tensorflow-hub |
>=0.9.0,<0.14 |
>=0.9.0,<0.13 |
|
pyarrow |
>=10,<11 |
>=6,<7 |
|
apache-beam |
>=2.40,<3 |
>=2.47,<3 |
|
scikit-learn |
>=1.0,<2 |
>=0.23,<0.24 |
|
google-api-core |
<3 |
<1.33 |
|
google-cloud-aiplatform |
>=1.6.2,<2 |
>=1.6.2,<1.18 |
|
tflite-support |
>=0.4.3,<0.4.5 |
>=0.4.2,<0.4.3 |
|
pyyaml |
>=6,<7 |
>=3.12,<6 |
Issue with installation of PyYaml 5.4.1. (https://github.com/yaml/pyyaml/issues/724) |
tensorflow |
>=2.13,<2.14 |
>=2.12,<2.13 |
|
tensorflowjs |
>=4.5,<5 |
>=3.6.0,<4 |
Placeholder
(and _PlaceholderOperator
) are no longer Jsonable
.Package Name | Version Constraints | Previously (in v1.13.0 ) |
Comments |
---|---|---|---|
tensorflow-hub |
>=0.9.0,<0.14 |
>=0.9.0,<0.13 |
|
pyarrow |
>=10,<11 |
>=6,<7 |
|
apache-beam |
>=2.40,<3 |
>=2.47,<3 |
|
scikit-learn |
>=1.0,<2 |
>=0.23,<0.24 |
|
google-api-core |
<3 |
<1.33 |
|
google-cloud-aiplatform |
>=1.6.2,<2 |
>=1.6.2,<1.18 |
|
tflite-support |
>=0.4.3,<0.4.5 |
>=0.4.2,<0.4.3 |
|
pyyaml |
>=6,<7 |
>=3.12,<6 |
Issue with installation of PyYaml 5.4.1. (https://github.com/yaml/pyyaml/issues/724) |
tensorflow |
>=2.13,<2.14 |
>=2.12,<2.13 |
|
tensorflowjs |
>=4.5,<5 |
>=3.6.0,<4 |
Conditional can be used from tfx.dsl.Cond
(Given from tfx import v1 as tfx
).
Dummy channel for testing can be constructed by
tfx.testing.Channel(artifact_type)
.
placeholder.Placeholder.placeholders_involved()
was replaced with
placeholder.Placeholder.traverse()
.
placeholder.Predicate.dependent_channels()
was replaced with
channel_utils.get_dependent_channels(Placeholder)
.
placeholder.Predicate.encode_with_keys(...)
was replaced with
channel_utils.encode_placeholder_with_channels(Placeholder, ...)
.
placeholder.Predicate.from_comparison()
removed (was deprecated)
enable external_pipeline_artifact_query
for querying artifact within one pipeline
Support InputArtifact[List[Artifact]]
annotation in Python function custom component
Package Name | Version Constraints | Previously (in v1.12.0 ) |
Comments |
---|---|---|---|
click |
>=7,<9 |
>=7,<8 |
|
ml-metadata |
~=1.13.1 |
~=1.12.0 |
Synced release train |
protobuf |
>=3.13,<4 |
>=3.20.3,<5 |
To support TF 2.12 |
struct2tensor |
~=0.44.0 |
~=0.43.0 |
Synced release train |
tensorflow |
~=2.12.0 |
>=1.15.5,<2 or ~=2.11.0 |
|
tensorflow-data-validation |
~=1.13.0 |
~=1.12.0 |
Synced release train |
tensorflow-model-analysis |
~=0.44.0 |
~=0.43.0 |
Synced release train |
tensorflow-transform |
~=1.13.0 |
~=1.12.0 |
Synced release train |
tfx-bsl |
~=1.13.0 |
~=1.12.0 |
Synced release train |
Conditional can be used from tfx.dsl.Cond
(Given from tfx import v1 as tfx
).
Dummy channel for testing can be constructed by
tfx.testing.Channel(artifact_type)
.
placeholder.Placeholder.placeholders_involved()
was replaced with
placeholder.Placeholder.traverse()
.
placeholder.Predicate.dependent_channels()
was replaced with
channel_utils.get_dependent_channels(Placeholder)
.
placeholder.Predicate.encode_with_keys(...)
was replaced with
channel_utils.encode_placeholder_with_channels(Placeholder, ...)
.
placeholder.Predicate.from_comparison()
removed (was deprecated)
enable external_pipeline_artifact_query
for querying artifact within one pipeline
Package Name | Version Constraints | Previously (in v1.12.0 ) |
Comments |
---|---|---|---|
click |
>=7,<9 |
>=7,<8 |
|
ml-metadata |
~=1.13.1 |
~=1.12.0 |
Synced release train |
protobuf |
>=3.13,<4 |
>=3.20.3,<5 |
To support TF 2.12 |
struct2tensor |
~=0.44.0 |
~=0.43.0 |
Synced release train |
tensorflow |
~=2.12.0 |
>=1.15.5,<2 or ~=2.11.0 |
|
tensorflow-data-validation |
~=1.13.0 |
~=1.12.0 |
Synced release train |
tensorflow-model-analysis |
~=0.44.0 |
~=0.43.0 |
Synced release train |
tensorflow-transform |
~=1.13.0 |
~=1.12.0 |
Synced release train |
tfx-bsl |
~=1.13.0 |
~=1.12.0 |
Synced release train |
Package Name | Version Constraints | Previously (in v1.11.0 ) |
Comments |
---|---|---|---|
tensorflow |
~=2.11.0 |
>=1.15.5,<2 or ~=2.10.0 |
|
tensorflow-decision-forests |
>=1.0.1,<2 |
==1.0.1 |
Make it compatible with more TF versions. |
ml-metadata |
~=1.12.0 |
~=1.11.0 |
Synced release train |
struct2tensor |
~=0.43.0 |
~=0.42.0 |
Synced release train |
tensorflow-data-validation |
~=1.12.0 |
~=1.11.0 |
Synced release train |
tensorflow-model-analysis |
~=0.43.0 |
~=0.42.0 |
Synced release train |
tensorflow-transform |
~=1.12.0 |
~=1.11.0 |
Synced release train |
tfx-bsl |
~=1.12.0 |
~=1.11.0 |
Synced release train |
Package Name | Version Constraints | Previously (in v1.11.0 ) |
Comments |
---|---|---|---|
tensorflow |
~=2.11.0 |
>=1.15.5,<2 or ~=2.10.0 |
|
tensorflow-decision-forests |
>=1.0.1,<2 |
==1.0.1 |
Make it compatible with more TF versions. |
ml-metadata |
~=1.12.0 |
~=1.11.0 |
Synced release train |
struct2tensor |
~=0.43.0 |
~=0.42.0 |
Synced release train |
tensorflow-data-validation |
~=1.12.0 |
~=1.11.0 |
Synced release train |
tensorflow-model-analysis |
~=0.43.0 |
~=0.42.0 |
Synced release train |
tensorflow-transform |
~=1.12.0 |
~=1.11.0 |
Synced release train |
tfx-bsl |
~=1.12.0 |
~=1.11.0 |
Synced release train |
This is the last version that supports TensorFlow 1.15.x. TF 1.15.x support will be removed in the next version. Please check the TF2 migration guide to migrate to TF2.
Artifact/Channel properties now support the new MLMD PROTO property type.
Supports environment variables in the placeholder expression. This placeholder can be used to generate beam_pipeline_args dynamically.
tflite-support
related dependencies from [examples]
to a separate
[tflite-support]
extra.flax
related dependencies from [examples]
to a separate [flax]
extra.Package Name | Version Constraints | Previously (in v1.8.0 ) |
Comments |
---|---|---|---|
tensorflow |
>=1.15.5,<2 or ~=2.10.0 |
>=1.15.5,<2 or ~=2.9.0 |
|
tflite-support |
~=0.4.2 |
>=0.1.0a1,<0.2.1 |
Update to a TF-2.10 compatible version. |
google-cloud-aiplatform |
>=1.6.2,<1.18 |
>=1.6.2,<2 |
Added to help pip dependency resolution. |
ml-metadata |
~=1.11.0 |
~=1.10.0 |
Synced release train |
struct2tensor |
~=0.42.0 |
~=0.41.0 |
Synced release train |
tensorflow-data-validation |
~=1.11.0 |
~=1.10.0 |
Synced release train |
tensorflow-model-analysis |
~=0.42.0 |
~=0.41.0 |
Synced release train |
tensorflow-transform |
~=1.11.0 |
~=1.10.0 |
Synced release train |
tfx-bsl |
~=1.11.0 |
~=1.10.0 |
Synced release train |