Coach Versions Save

TD3 New APIs for Coach usage as a library Updated Getting Started tutorial Batch RL tutorial

Fixes for breaking API changes (OpenAI Gym, Scipy) OPE: Weighted Importance Sampling Creating a dataset using an agent Printing input size as part of network summary

ACER Soft Actor-Critic BCQ Batch RL Off-policy evaluation (estimators: DM, DR, Sequential DR, IPS)

Intel Tensorflow fix.

Roll out worker memory leak fix wxPython dependency removal

Horizontal scaling MxNet support ONNX export New documentation

A complete redesign - non-backward compatible. Enabling multi-agent support.

New features -

• PIP package
• Benchmarks
• Hierarchical Reinforcement Learning (demonstrated by Hierarchical Actor-Critic)
• Tutorials
• Shared memory (e.g. Replay Buffer) between workers
• Tests (unit-tests, reward-based tests, trace-based tests)
• Using Coach as a library (see example here)

New Environments -

• Toy Environments (Exploration Chain, BitFlip)
• DeepMind PySC2 support (Starcraft 2)
• DeepMind Control Suite

New Algorithms -

• Hindsight Experience Replay
• Prioritized Experience Replay
• Hierarchical Actor-Critic
• UCB with Q-Ensembles

New features -

• CARLA 0.7 simulator integration
• Human control of the game play
• Recording of human game play and storing / loading the replay buffer
• Behavioral cloning agent and presets
• Golden tests for several presets
• Selecting between deep / shallow image embedders
• Rendering through pygame (with some boost in performance)

API changes -

• Improved environment wrapper API
• Added an evaluate flag to allow convenient evaluation of existing checkpoints
• Improve frameskip definition in Gym

Bug fixes -

• Fixed loading of checkpoints for agents with more than one network
• Fixed the N Step Q learning agent python3 compatibility

Initial public release