CommonLoopUtils Versions Save

What's Changed

• Switch from jax.tree_map (deprecated since JAX 0.4.26) to jax.tree_util.tree_map.
• Improvements to parameter overview.

Full Changelog: https://github.com/google/CommonLoopUtils/compare/v0.0.11...v0.0.12

What's Changed

• Removes numpy version pin
• Adds sharding annotations, dtype, total bytes to parameter_overview
• Makes clu.metrics.Std support same shapes as clu.metrics.Average

Full Changelog: https://github.com/google/CommonLoopUtils/compare/v0.0.10...v0.0.11

This release requires Python>=3.10

What's Changed

• clu.parameter_overview now supports JAX global arrays.
• Various small fixes in clu.metrics module.
• Removed s

Full Changelog: https://github.com/google/CommonLoopUtils/compare/v0.0.9...v0.0.10

Last release before dropping support for Python 3.7 and 3.8

What's Changed

• Fix pytype failures related to teaching pytype about NumPy scalar types.
• Fix a couple of docstring typos.
• Updates README and clu_synposis.ipynb

Full Changelog: https://github.com/google/CommonLoopUtils/compare/v0.0.8...v0.0.9

• Moves clu.internal.asynclib to clu.asynclib.
• Adds methods for writing raw tensors and audio to MetricWriter.
• Adds clu.values to annotate arrays with a modality.
• Adds clu.data.DatasetIterator - a generic interface between input pipelines and training loops.
• Fixes various issues with clu.metrics.

This will be the last release supporting Python 3.7.

• Makes deterministic_data work with tfds>4.4.0 and tfds<=4.4.0.

• Log error instead of failing when profiler.start() raises an exception.
• Makes periodic_actions.ProgressUpdate show total number of steps.
• Makes AsyncWriter non-blocking wrt JAX async computations.
• Adds clu_synopsis.ipynb Colab as initial documentation.
• Restore Checkpoint without providing the state
• Makes PreprocessFn addable.
• Allow n-dimensional arrays (and masks) to be passed to Metrics.Average().

deterministic_data

• Support non-positive input value for pad_up_to_batches.
• Support padding dataset when data dimension is unknown.
• Support TFDS specs in get_read_instruction_for_host.
• Allow changing drop_remainder for batching.
• Add RemainderOptions in deterministic_data.

metric_writers

• Support multiple writers in metric_writers.ensure_flushes.

metrics

• Makes internal.flatten_dict() work with ConfigDicts.
• Forwards mask model output to metrics created via Metric.from_output().
• Forwards mask model output to metrics created via Metric.from_fun().
• Added Collections.unreplicate(), Collections.create().

periodic_actions

• Formats long time strings in '{days}d{hours}h{mins}m' format.

preprocess_spec

• Make feature description of features in PreprocessFn more compact.
• Better type check in preprocess_spec.get_all_ops().

Documentation:

• Added clu_synopsis.ipynb Colab

Changelog:

• metric_writers: Lets SummaryWriter write nested dictionaries.
• internal: Adds async.Pool.
• preprocess_spec: Support nested dictionaries.
• profile: Use JAX profiler APIs instead of TF profiler APIs.