ML-Agents Release 11

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.7.0
ml-agents (Python)	v0.23.0
ml-agents-envs (Python)	v0.23.0
gym-unity (Python)	v0.23.0
Communicator (C#/Python)	v1.3.0

Major Features and Improvements

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• An individual agent can now take both continuous and discrete actions. You can specify both continuous and discrete action sizes in Behavior Parameters. (#4702, #4718)

ml-agents / ml-agents-envs / gym-unity (Python)

• PyTorch trainers now support training agents with both continuous and discrete action spaces. (#4702)

Bug Fixes and Minor Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• In order to improve the developer experience for Unity ML-Agents Toolkit, we have added in-editor analytics. Please refer to "Information that is passively collected by Unity" in the Unity Privacy Policy. (#4677)
• The FoodCollector example environment now uses continuous actions for moving and discrete actions for shooting. (#4746)
• Removed noisy warnings about API minor version mismatches in both the C# and python code. (#4688)

ml-agents / ml-agents-envs / gym-unity (Python)

• ActionSpec._validate_action() now enforces that UnityEnvironment.set_action_for_agent() receives a 1D np.array.

ML-Agents Release 10

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.6.0
ml-agents (Python)	v0.22.0
ml-agents-envs (Python)	v0.22.0
gym-unity (Python)	v0.22.0
Communicator (C#/Python)	v1.2.0

Major Features and Improvements

New Demo Environment

• The Match3 environment was added to the Project, it uses the new utilities added in com.unity.ml-agents.extensions.

ml-agents / ml-agents-envs / gym-unity (Python)

• PyTorch trainers are now the default. See the installation docs for more information on installing PyTorch. For the time being, TensorFlow is still available; you can use the TensorFlow backend by adding --tensorflow to the CLI, or adding framework: tensorflow in the configuration YAML. (#4517)

Bug Fixes and Minor Changes

com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

• The Barracuda dependency was upgraded to 1.1.2 (#4571)
• Utilities were added to com.unity.ml-agents.extensions to make it easier to integrate with match-3 games. See the readme for more details. (#4515)
• Agent.CollectObservations() and Agent.EndEpisode() will now throw an exception if they are called recursively (for example, if they call Agent.EndEpisode()). Previously, this would result in an infinite loop and cause the editor to hang. (#4573)

ml-agents / ml-agents-envs / gym-unity (Python)

• The action_probs node is no longer listed as an output in TensorFlow models (#4613).

ML-Agents Release 9

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.5.0
ml-agents (Python)	v0.21.1
ml-agents-envs (Python)	v0.21.1
gym-unity (Python)	v0.21.1
Communicator (C#/Python)	v1.2.0

Bug Fixes and Minor Changes

ml-agents (Python)

• Fixed an issue where runs could not be resumed when using TensorFlow and Ghost Training. (#4593)
• Capped cattrs version to 1.0.x. (#4613)
• Capped PyTorch version to 1.6.x. (#4617)

ML-Agents Release 8

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.5.0
ml-agents (Python)	v0.21.0
ml-agents-envs (Python)	v0.21.0
gym-unity (Python)	v0.21.0
Communicator (C#/Python)	v1.2.0

Major Features and Improvements

com.unity.ml-agents (C#)

• Stacking for compressed observations is now supported. An additional setting option Observation Stacks is added to the sensor components that support compressed observations. A new class ISparseChannelSensor with an additional method GetCompressedChannelMapping()is added to generate a mapping of the channels in compressed data to the actual channel after decompression for the python side to decompress correctly. (#4476)

ml-agents (Python)

• Added the Random Network Distillation (RND) intrinsic reward signal to the Pytorch trainers. To use RND, add a rnd section to the reward_signals section of your yaml configuration file. More information here. (#4473)
• The Communication API was changed to 1.2.0 to indicate support for stacked compressed observation. A new entry compressed_channel_mapping is added to the proto to handle decompression correctly. Newer versions of the package that wish to make use of this will also need a compatible version of the Python trainers. (#4476)

Bug Fixes and Minor Changes

com.unity.ml-agents (C#)

• Fixed a bug where accessing the Academy outside of play mode would cause the Academy to get stepped multiple times when in play mode. (#4532)

ml-agents (Python)

• In the VisualFoodCollector scene, a vector flag representing the frozen state of the agent is added to the input observations in addition to the original first-person camera frame. The scene is able to train with the provided default config file. (#4511)
• Added a new visual scene Visual3DBall in the 3DBall example. (#4513)
• Added conversion to string for sampler classes to increase the verbosity of the curriculum lesson changes. The lesson updates would now output the sampler stats in addition to the lesson and parameter name to the console. (#4484)
• Localized documentation in Russian is added. (#4529)

Acknowledgements

Thank you @SergeyMatrosov for your contributions to this release.

ML-Agents Release 7

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.4.0
ml-agents (Python)	v0.20.0
ml-agents-envs (Python)	v0.20.0
gym-unity (Python)	v0.20.0
Communicator (C#/Python)	v1.1.0

Major Features and Improvements

com.unity.ml-agents (C#)

• The IActuator interface and ActuatorComponent abstract class were added. These are analogous to ISensor and SensorComponent, but for applying actions for an Agent. They allow you to control the action space more programmatically than defining the actions in the Agent's Behavior Parameters. See BasicActuatorComponent.cs for an example of how to use them. (#4297, #4315)

ml-agents (Python)

• Experimental PyTorch support has been added. Use --torch when running mlagents-learn, or add framework: pytorch to your trainer configuration (under the behavior name) to enable it. Note that PyTorch 1.6.0 or greater should be installed to use this feature; see see the PyTorch website for installation instructions and the relevant ML-Agents docs for usage. (#4335)

Breaking Changes

ml-agents (Python)

The minimum supported version of TensorFlow was increased to 1.14.0. (#4411)

Known Issues

ml-agents (Python)

• Soft-Actor Critic (SAC) runs considerably slower when using the PyTorch backend than when using TensorFlow.

Bug Fixes and Minor Changes

com.unity.ml-agents (C#)

• Updated Barracuda to 1.1.1-preview (#4482)
• Enabled C# formatting using dotnet-format. (#4362)
• GridSensor was added to the com.unity.ml-agents.extensions package. Thank you to Jaden Travnik from Eidos Montreal for the contribution! (#4399)
• Added Agent.EpisodeInterrupted(), which can be used to reset the agent when it has reached a user-determined maximum number of steps. This behaves similarly to Agent.EndEpsiode() but has a slightly different effect on training (#4453).
• Previously, com.unity.ml-agents was not declaring built-in packages as dependencies in its package.json. The relevant dependencies are now listed. (#4384)
• Fixed the sample code in the custom SideChannel example. (#4466)

ml-agents (Python)

• Compressed visual observations with >3 channels are now supported. In ISensor.GetCompressedObservation(), this can be done by writing 3 channels at a time to a PNG and concatenating the resulting bytes. (#4399)
• The Communication API was changed to 1.1.0 to indicate support for concatenated PNGs (see above). Newer versions of the package that wish to make use of this will also need a compatible version of the trainer. (#4462)
• A CNN (vis_encode_type: match3) for smaller grids, e.g. board games, has been added. (#4434)
• You can now again specify a default configuration for your behaviors. Specify default_settings in your trainer configuration to do so. (#4448)
• A bug in the observation normalizer that would cause rewards to decrease when using --resume was fixed. (#4463)

Acknowledgements

Thank you @NeonMika , @armando-fandango, @Sebastian-Schuchmann , and everyone at Unity for their contributions to this release.

ML-Agents Release 6

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.3.0
ml-agents (Python)	v0.19.0
ml-agents-envs (Python)	v0.19.0
gym-unity (Python)	v0.19.0
Communicator (C#/Python)	v1.0.0

Breaking Changes

ml-agents (Python)

• The minimum supported Python version for ml-agents-envs was changed to 3.6.1. (#4244)

Bug Fixes and Minor Changes

com.unity.ml-agents (C#)

• Academy.EnvironmentStep() will now throw an exception if it is called recursively (for example, by an Agent's CollectObservations method). Previously, this would result in an infinite loop and cause the editor to hang. (#1551)

ml-agents (Python)

• The supported versions of numpy ml-agents-envs was changed to not allow 1.19.0 or later. This was done to reflect a similar change in TensorFlow's requirements. (#4274)
• Model checkpoints are now also saved as .nn files during training. (#4127)
• Model checkpoint info is saved in TrainingStatus.json after training is concluded (#4127)
• The CSV statistics writer was removed (#4300).
• StatsSideChannel now stores multiple values per key. This means that multiple calls to StatsRecorder.Add() with the same key in the same step will no longer overwrite each other. (#4236)

ml-agents-envs (Python)

• The interaction between EnvManager and TrainerController was changed; EnvManager.advance() was split into two stages, and TrainerController now uses the results from the first stage to handle new behavior names. This change speeds up Python training by approximately 5-10%. (#4259)

Known Issues

com.unity.ml-agents (C#)

• On macOS 10.15, if the ML-Agents package is installed from a website download, you may receive a “file cannot be opened issue” when running scenes that use ML-Agents. Workarounds include installing the package using the Unity Package Manager, or following the instructions here. (specifically, the section titled “How to open an app that hasn’t been notarized or is from an unidentified developer”).

Acknowledgements

Thank you ChristianCoenen, niskander, and everyone at Unity for their contributions to this release.

Package Versions

As part of ML-Agents Release 5, we will be versioning the different packages that make up the release.

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.2.0
ml-agents (Python)	v0.18.1
ml-agents-envs (Python)	v0.18.1
gym-unity (Python)	v0.18.1
Communicator (C#/Python)	v1.0.0

Bug Fixes

ml-agents / ml-agents-envs / gym-unity (Python)

• Summary writer no longer crashes if Hyperparameters could not be written (#4265)
• Reduce numpy version in ml-agents-envs setup (#4274)
• Fix curriculum lesson incrementing past final lesson (#4279)

Package Versions

NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.2.0
ml-agents (Python)	v0.18.0
ml-agents-envs (Python)	v0.18.0
gym-unity (Python)	v0.18.0
Communicator (C#/Python)	v1.0.0

Major Features and Improvements

ml-agents (Python)

• The Parameter Randomization feature has been refactored and merged with the Curriculum. This enables sampling of new parameters per episode to improve robustness. The resampling-interval parameter has been removed and the config structure updated. More information here. (#4065)
• Curriculum has been refactored and is now specified at the level of the parameter, not the behavior. It is also now possible to specify a sampler in the lesson of a Curriculum. More information here.(#4160)

Breaking Changes

ml-agents (Python)

• The training configuration yaml file format has changed. In order to use curriculum or environment randomization, you must update your configuration file with the new section environment_parameters. We provide an update script to help this migration. Run python -m mlagents.trainers.upgrade_config -h with the latest version to see the script usage.

Bug Fixes and Minor Changes

com.unity.ml-agents (C#)

• SideChannelsManager was renamed to SideChannelManager. The old name is still supported, but deprecated. (#4137)
• RayPerceptionSensor.Perceive() now additionally stores the GameObject that was hit by the ray. (#4111)
• Fixed an issue where RayPerceptionSensor would raise an exception when the list of tags was empty, or a tag in the list was invalid (unknown, null, or empty string). (#4155)
• The Barracuda dependency was upgraded to 1.0.1 (#4188)

ml-agents-envs (Python)

• Added new Google Colab notebooks to show how to use UnityEnvironment. (#4117)

ml-agents (Python)

• Fixed an error when setting initialize_from in the trainer configuration YAML to null. (#4175)
• Fixed a rare crash in StatsReporter when using threaded trainers (#4201)

Example Environments (C#)

• Fixed issue with FoodCollector, Soccer, and WallJump when playing with keyboard. (#4147, #4174)

Known Issues

com.unity.ml-agents (C#)

• On macOS 10.15, if the ML-Agents package is installed from a website download, you may receive a “file cannot be opened issue” when running scenes that use ML-Agents. Workarounds include installing the package using the Unity Package Manager, or following the instructions here.

Acknowledgements

Thank you @StefanoCecere, @yongjun823, @poehlerflorian, @ChristianCoenen, @textarcana, @furkan-celik and everyone at Unity for their contributions to this release.

Package Versions

As part of ML-Agents Release 3, we will be versioning the different packages that make up the release. NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.1.0
ml-agents (Python)	v0.17.0
ml-agents-envs (Python)	v0.17.0
gym-unity (Python)	v0.17.0
Communicator (C#/Python)	v1.0.0

Major Features and Improvements

com.unity.ml-agents (C#)

• ObservableAttribute was added. Adding the attribute to fields or properties on an Agent will allow it to generate observations via reflection. (#3925, #4006)

ml-agents-envs (Python)

• First version of the Unity Environment Registry (Experimental) has been released. The Environment Registry allows running pre-built Unity environments from Python without installing the Unity editor. More information here (#3967)

gym-unity (Python)

• use_visual and allow_multiple_visual_obs in the UnityToGymWrapper constructor were replaced by allow_multiple_obs which allows one or more visual observations and vector observations to be used simultaneously. (#3981) Thank you @shakenes !

ml-agents (Python)

• When using a Curriculum, the current lesson will resume if training is quit and resumed. As such, the --lesson CLI option has been removed. (#4025)
• In PPO training, beta and epsilon now follow the same decay schedule as learning_rate. Previously they were always decayed. (#3940)

Example Environments

• Added new Walker environments. Improved ragdoll stability/performance. (#4037)

Breaking Changes

ml-agents

• Format for trainer configuration has changed, and the "default" behavior has been deprecated. See the Migration Guide for more details. (#3936)
• Curriculum and Parameter Randomization configurations have been merged into the main training configuration file. Note that this means training configuration files are now environment-specific. (#3791)
• Training artifacts (trained models, summaries) are now found in the results/ directory. (#3829) --save-freq CLI option has been removed, and replaced by a checkpoint_interval option in the trainer configuration YAML. (#4034)

ml-agents-envs

• get_behavior_names() and get_behavior_spec() on UnityEnvironment were replaced by the behavior_specs property. (#3946)
• In the Python API, max_step in the TerminalStep and TerminalSteps objects was renamed interrupted. (#3908)

Bug Fixes and Minor Changes

ml-agents

• Unity Player logs are now written out to the results directory. (#3877)
• Run configuration YAML files are written out to the results directory at the end of the run. (#3815)
• When trying to load/resume from a checkpoint created with an earlier version of ML-Agents, a warning will be thrown. (#4035)
• Fixed an issue where SAC would perform too many model updates when resuming from a checkpoint, and too few when using buffer_init_steps. (#4038)

Acknowledgements

Thank you @shakenes, @TPihko, and everyone at Unity for their contributions to this release.

Package Versions

As part of ML-Agents Release 2, we will be versioning the different packages that make up the release. NOTE: It is strongly recommended that you use packages from the same release together for the best experience.

Package	Version
com.unity.ml-agents (C#)	v1.0.2
ml-agents (Python)	v0.16.1
ml-agents-envs (Python)	v0.16.1
gym-unity (Python)	v0.16.1
Communicator (C#/Python)	v1.0.0

Bug Fixes

com.unity.ml-agents (C#)

• A bug that would cause the editor to go into a loop when a prefab was selected was fixed. (#3949)
• BrainParameters.ToProto() no longer throws an exception if none of the fields have been set. (#3930)
• The Barracuda dependency was upgraded to 0.7.1-preview. (#3977)

ml-agents / ml-agents-envs / gym-unity (Python)

• An issue was fixed where using --initialize-from would resume from the past step count. (#3962)
• The gym wrapper error for the wrong number of agents now fires more consistently, and more details were added to the error message when the input dimension is wrong. (#3963)
• Miscellaneous documentation fixes.