A framework for easy prototyping of distributed reinforcement learning algorithms
distributedRL is a framework for prototyping disrtibuted reinforcement learning using Pytorch, Ray, and ZeroMQ (and more). You can make use of the structural components to easily distribute your reinforcement learning algorithm, on a single-machine.
Currently, an implementation of Ape-X DQN is available. IMPALA and SEED are planned to be implemented.
Feel free to reach out (cjy2129 at columbia dot edu) or raise an issue if you have any questions!
ApeXLearner
and ApeXWorker
are implemented as abstract classes (that inherit common.abstract.Learner
and common.abstract.Worker
respectively). To extend Ape-X to any off-policy RL algorithm, you just have to implement algorithm-specific details, such as action selection, environment step, and learning step (check the abstract classes for more detail).clone the repository, then
conda env create -f environment.yml
conda activate distrl
pip install -e .
To be added
To be added
I thank @Avrech for his helpful discussion, suggestions and enhancements. In particular, on identifying bugs and bottlenecks and improving asynchrony of the Ape-X implementation.