Major Features and Improvements

• Add F1, False positive rate, and Accuracy into the confusion matrix plot.
• Add support for setting top_k and class_id at the same time for confusion matrix metrics.
• Add the false positive for semantic segmentation metrics.
• Add Mean Metric (experimental) which calculates the mean of any feature. *. Adds support of output_keypath to ModelSignatureDoFn to explicitly set a chain of output keys in the multi-level dict (extracts). Adds output_keypath to common prediction extractors.

Bug fixes and other Changes

• Fix the bug that SetMatchRecall is always 1 when top_k is set.
• Add 'tfma_eval' model_type in model_specs as the identifier for eval_saved_model, allowing signature='eval' to now be used with other model types.
• Add "materialized_prediction" model type to allow users bypassing model inference explicitly.
• Depends on pyarrow>=10,<11.
• Depends on apache-beam>=2.47,<3.
• Depends on numpy>=1.23.0.
• Depends on tensorflow>=2.13.0,<3.

Breaking Changes

• Depend on PIL for image related metrics.
• Separate extract_key from signature names in ModelSignaturesDoFn.

Deprecations

• N/A

Major Features and Improvements

• Add BinaryCrossEntropy and CategoricalCrossEntropy.
• Add MeanAbsolutePercentageError and MeanSquaredLogarithmicError
• Add SetMatchPrecision and SetMatchRecall

Bug fixes and other Changes

• Fix for jupiter notebook
• Fix element dimension inconsistency when some of the extracts have missing key.
• Add public visibility to the servo beam extractor.
• Fix for bug where binary_confusion_matrices with different class_weights are considered identical and deduplicated.
• Fixes bug where positive / negative axes labels are reversed in prediction distribution plot.
• Depends on numpy~=1.22.0.
• Modify ExampleCount to not depend on labels and predictions.
• Add class_id info into sub_key in metric_key for object detection confusion matrix metrics.
• Add class_id info into sub_key in plot_key for object detection confusion matrix plot.
• Fix a bug that auto_pivot dropped nan when deciding which columns are multivalent for pivoting.
• Depends on tensorflow>=2.12.0,<2.13.
• Depends on protobuf>=3.20.3,<5.
• Depends on tfx-bsl>=1.13.0,<1.14.0.
• Depends on tensorflow-metadata>=1.13.1,<1.14.0.

Breaking Changes

• N/A

Deprecations

Deprecated python3.7 support.

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Depends on tensorflow>=2.11,<3
• Depends on tfx-bsl>=1.2.0,<1.13.0.
• Depends on tensorflow-metadata>=1.12.0,<1.13.0.

Breaking Changes

• N/A

Deprecations

• N/A

Major Features and Improvements

• This is the last version that supports TensorFlow 1.15.x. TF 1.15.x support will be removed in the next version. Please check the TF2 migration guide to migrate to TF2.
• Add BooleanFlipRate metric for comparing thresholded predictions between multiple models.
• Add CounterfactualPredictionsExtractor for computing predictions on modified inputs.

Bug fixes and other Changes

• Add support for parsing the Predict API prediction log output to the experimental TFX-BSL PredictionsExtractor implementation.

• Add support for parsing the Classification API prediction log output to the experimental TFX-BSL PredictionsExtractor implementation.

• Update remaining predictions_extractor_test.py tests to cover PredictionsExtractorOSS. Fixes a pytype bug related to multi tensor output.

• Depends on tensorflow>=1.15.5,<2 or tensorflow>=2.10,<3

• Apply changes in the latest Chrome browser

• Add InferneceInterface to experimental PredictionsExtractor implementation.

• Stop returning empty example_ids metric from binary_confusion_matrices derived computations when example_id_key is not set but use_histrogam is true.

• Add transformed features lookup for NDCG metrics query key and gain key.

• Deprecate BoundedValue and TDistribution in ConfusionMatrixAtThresholds.

• Fix a bug that dataframe auto_pivot fails if there is only Overall slice.

• Use SavedModel PB to determine default signature instead of loading the model.

• Reduce clutter in the multi-index columns and index in the experimental dataframe auto_pivot util.

Breaking Changes

• N/A

Deprecations

• N/A

Version 0.41.1

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Move the version to top of init.py since the original "from tensorflow_model_analysis.sdk import *" will not import private symbol.

Breaking Changes

• N/A

Deprecations

• N/A

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Move the version to top of init.py since the original "from tensorflow_model_analysis.sdk import *" will not import private symbol.

Breaking Changes

• N/A

Deprecations

• N/A

Major Features and Improvements

• Add COCO object detection metrics, object detection related utilities, objection detection opitons in binary confusion matrix, Precision At Recall, and AUC. Add MaxRecall metric.
• Add support for parsing sparse tensors with explicit tensor representations via TFXIO.

Bug fixes and other Changes

• Add score_distribution_plot.
• Separate the Predictions Extractor into two extractors.
• Update PredictionsExtractor to support backwards compatibility with the Materialized Predictions Extractor.
• Depends on apache-beam[gcp]>=2.40,<3.
• Depends on pyarrow>=6,<7.
• Update merge_extracts with an option to skip squeezing one-dim arrays. Update split_extracts with an option to expand zero-dim arrays.
• Added experimental bulk inference implementation to PredictionsExtractor. Currently only supports the RegressionAPI.

Breaking Changes

• Adds multi-index columns for view.experimental.metrics_as_dataframe util.
• Changes SymmetricPredictionDifference output type from array to scalar.

Deprecations

• N/A

Major Features and Improvements

• Add object detection related utilities.

Bug fixes and other Changes

• Depends on tensorflow>=1.15.5,<2 or tensorflow>=2.9,<3
• Fix issue where labels with -1 values are one-hot encoded when they shouldn't be ## Breaking Changes
• Depends on tfx-bsl>=1.9.0,<1.10.0.
• Depends on tensorflow-metadata>=1.9.0,<1.10.0.

Breaking Changes

• N/A

Deprecations

• N/A

Major Features and Improvements

• SqlSliceKeyExtractor now supports slicing on transformed features.

Bug fixes and other Changes

• Depends on tfx-bsl>=1.8.0,<1.9.0.
• Depends on tensorflow-metadata>=1.8.0,<1.9.0.
• Depends on apache-beam[gcp]>=2.38,<3.

Breaking Changes

• N/A

Deprecations

• N/A

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Fixes issue attempting to parse metrics, plots, and attributions without a format suffix.
• Fixes the non-deterministic key ordering caused by proto string serialization in metrics validator.
• Update variable name to respectful terminology, rebuild JS
• Fixes issues preventing standard preprocessors from being applied.
• Allow merging extracts including sparse tensors with different dense shapes.

Breaking Changes

• MetricsPlotsAndValidationsWriter will now write files with an explicit output format suffix (".tfrecord" by default). This should only affect pipelines which directly construct MetricsPlotsAndValidationWriter instances and do not set output_file_format. Those which use default_writers() should be unchanged.
• Batched based extractors previously worked off of either lists of dicts of single tensor values or arrow record batches. These have been updated to be based on dicts with batched tensor values at the leaves.
• Depends on tensorflow>=1.15.5,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,<3.
• Depends on tfx-bsl>=1.7.0,<1.8.0.
• Depends on tensorflow-metadata>=1.7.0,<1.8.0.
• Depends on apache-beam[gcp]>=2.36,<3.

Deprecations

• N/A

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Fixed issue with aggregation type not being set properly in keys associated with confusion matrix metrics.
• Enabled the sql_slice_key extractor when evaluating a model.
• Depends on numpy>=1.16,<2.
• Depends on absl-py>=0.9,<2.0.0.
• Depends on tensorflow>=1.15.5,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,<3.
• Depends on tfx-bsl>=1.6.0,<1.7.0.
• Depends on tensorflow-metadata>=1.6.0,<1.7.0.
• Depends on apache-beam[gcp]>=2.35,<3.

Breaking Changes

• N/A

Deprecations

• N/A