RL_CARLA

SAC in Carla simulator

Based on PARL and Torch/Paddle(Baidu deep learning framework), a parallel version of SAC was implemented and achieved high performance in the CARLA environment.

Paper: SAC in Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Carla simulator introduction

Please see Carla simulator to know more about Carla simulator.

Benchmark result

• Result was evaluated with mode Lane

How to use

• System: Ubuntu 16.04

Dependencies:

• Simulator: CARLA
• RL env: gym_carla

Installation

1. Create conda environment

   $ conda create -n rl_carla python=3.6
   $ conda activate rl_carla
   

2. Download CARLA_0.9.6, extract it to some folder, and add CARLA to PYTHONPATH environment variable

   $ export PYTHONPATH="SOMEFOLDER/CARLA_0.9.6/PythonAPI/carla/dist/carla-0.9.6-py3.5-linux-x86_64.egg:$PYTHONPATH"
   

3. Clone repository from this repo

   $ git clone https://github.com/ShuaibinLi/RL_CARLA.git
   

4. Install the packages

   ## install requirements，
   ## Install paddle or torch wrt which base you are using(paddle or torch)
   ## Make sure only one deep-learning framework exists in your conda_env during traing
   $ pip install -r requirements.txt
   
   ## install gym env of carla
   $ cd gym_carla
   $ pip install -e .
   

   or you can install the package that you need by pip/conda install [package name]

Start Training

1. Open another(new) terminal, enter the CARLA root folder and launch CARLA service. There are two modes to start the CARLA server:
   (1) non-display mode

   $ DISPLAY= ./CarlaUE4.sh -opengl -carla-port=2021
   

   (2) display mode

   $ ./CarlaUE4.sh -windowed -carla-port=2021
   

   • Start three CARLA services (ports: 2021,2023,2025) for data collecting and training, one service (port: 2027) for evaluating.

2. For parallel training, we can execute the following xparl command to start a PARL cluster：

   $ xparl start --port 8080
   

   check xparl cluster status by xparl status

3. Start training

   $ python train.py --xparl_addr localhost:8080
   

Evaluate trained agent

Open another(new) terminal, enter the CARLA root folder and launch CARLA service with display mode.

$ ./CarlaUE4.sh -windowed -carla-port=2029

Restore saved model to see performance.

$ python evaluate.py --restore_model model.ckpt