Carla Imitation Learning Trainer
Carla Imitation Learning Trainer
The main project belongs to Carla Team, and I just wrote a training system for the main project. Here you can find more information about the original project.
Please check out issue and issue to find out why this code was published.
TODO:
Please feel free to pull a request. Any features and any changes are welcome.
All Contributors:
This is an auto table generator for the paper writing purpose. Tables are based on the CoRL 2017 Carla Team Paper. This can help me to have a gorgeous table easier and with a lot of fewer efforts. Currently, Text, LaTex, and HTML are supported. Before you are going to run this code, you have to prepare the results for three different runs for each town in Carla. You can see an example in the following passage.
CarlaPaper_ReExperiment/
├── CarlaPaperModel_Test01_CoRL2017_Town01
│ ├── log_201807010542
│ ├── log_201807011830
│ ├── log_201807020224
│ ├── measurements.csv
│ ├── metrics.json
│ ├── res
│ └── summary.csv
├── CarlaPaperModel_Test01_CoRL2017_Town02
│ ├── log_201807031155
│ ├── log_201807032256
│ ├── log_201807050552
│ ├── measurements.csv
│ └── summary.csv
├── CarlaPaperModel_Test02_CoRL2017_Town01
│ ├── log_201807020250
│ ├── log_201807020739
│ ├── log_201807021521
│ ├── measurements.csv
│ └── summary.csv
├── CarlaPaperModel_Test02_CoRL2017_Town02
│ ├── log_201807050555
│ ├── measurements.csv
│ └── summary.csv
├── CarlaPaperModel_Test03_CoRL2017_Town01
│ ├── log_201807022050
│ ├── log_201807030243
│ ├── log_201807030738
│ ├── measurements.csv
│ └── summary.csv
├── CarlaPaperModel_Test03_CoRL2017_Town02
│ ├── log_201807052015
│ ├── log_201807060744
│ ├── measurements.csv
│ └── summary.csv
├── results.html
└── results.laTex
P.S: results.html and results.laTex are produced by this code after execution. These are a little bit different because of different configuration and different Carla version. In these results, bikes were in the test only. Please share your code and enhancements by a pull request with the world if you use this in your papers and add a feature.
MODEL: CoRL-2017 Carla Paper_ReExperiment
Tasks | Training Conditions | New Town | New Weather | New Town & Weather |
---|---|---|---|---|
Straight | 0.98 +/- 0.0 | 0.97 +/- 0.0 | 0.97 +/- 0.01 | 0.95 +/- 0.0 |
One Turn | 0.94 +/- 0.01 | 0.68 +/- 0.01 | 0.94 +/- 0.02 | 0.7 +/- 0.0 |
Navigation | 0.89 +/- 0.0 | 0.42 +/- 0.0 | 0.83 +/- 0.01 | 0.46 +/- 0.02 |
Nav. Dynamic | 0.88 +/- 0.0 | 0.42 +/- 0.02 | 0.8 +/- 0.03 | 0.46 +/- 0.03 |
Infractions | Training Conditions | New Town | New Weather | New Town & Weather |
---|---|---|---|---|
Opposite Lane | 16.78 +/- 0.04 | 2.89 +/- 0.0 | 8.13 +/- 0.13 | 3.89 +/- 0.5 |
Sidewalk | 7.46 +/- 1.32 | 2.65 +/- 0.34 | 8.13 +/- 0.13 | 2.26 +/- 0.2 |
Collision-static | 16.78 +/- 0.04 | 8.67 +/- 0.0 | 8.13 +/- 0.13 | 12.73 +/- 0.01 |
Collision-car | 16.78 +/- 0.04 | 8.67 +/- 0.0 | 8.13 +/- 0.13 | 12.73 +/- 0.01 |
Collision-pedestrian | 16.78 +/- 0.04 | 8.67 +/- 0.0 | 8.13 +/- 0.13 | 12.73 +/- 0.01 |
Infractions | Training Conditions | New Town | New Weather | New Town & Weather |
---|---|---|---|---|
Opposite Lane | 13.09 +/- 2.68 | 1.03 +/- 0.11 | 16.86 +/- 0.09 | 1.35 +/- 0.1 |
Sidewalk | 11.22 +/- 0.1 | 0.9 +/- 0.05 | 16.86 +/- 0.09 | 0.95 +/- 0.04 |
Collision-static | 33.66 +/- 0.3 | 17.68 +/- 0.26 | 16.86 +/- 0.09 | 26.88 +/- 0.7 |
Collision-car | 33.66 +/- 0.3 | 17.68 +/- 0.26 | 16.86 +/- 0.09 | 26.88 +/- 0.7 |
Collision-pedestrian | 33.66 +/- 0.3 | 17.68 +/- 0.26 | 16.86 +/- 0.09 | 26.88 +/- 0.7 |
Infractions | Training Conditions | New Town | New Weather | New Town & Weather |
---|---|---|---|---|
Opposite Lane | 17.4 +/- 3.7 | 2.26 +/- 0.05 | 23.47 +/- 8.47 | 3.12 +/- 0.17 |
Sidewalk | 16.27 +/- 4.14 | 0.62 +/- 0.03 | 12.71 +/- 3.75 | 0.68 +/- 0.03 |
Collision-static | 66.63 +/- 0.2 | 24.88 +/- 0.5 | 35.12 +/- 0.28 | 40.43 +/- 0.45 |
Collision-car | 66.63 +/- 0.2 | 24.88 +/- 0.5 | 35.12 +/- 0.28 | 40.43 +/- 0.45 |
Collision-pedestrian | 66.63 +/- 0.2 | 24.88 +/- 0.5 | 35.12 +/- 0.28 | 40.43 +/- 0.45 |
Infractions | Training Conditions | New Town | New Weather | New Town & Weather |
---|---|---|---|---|
Opposite Lane | 20.17 +/- 2.74 | 1.5 +/- 0.18 | 24.95 +/- 12.95 | 1.66 +/- 0.04 |
Sidewalk | 13.05 +/- 2.98 | 0.65 +/- 0.06 | 4.26 +/- 0.87 | 0.68 +/- 0.07 |
Collision-static | 4.51 +/- 1.07 | 0.38 +/- 0.05 | 2.15 +/- 0.55 | 0.41 +/- 0.05 |
Collision-car | 1.19 +/- 0.14 | 0.28 +/- 0.03 | 1.48 +/- 0.28 | 0.27 +/- 0.02 |
Collision-pedestrian | 14.14 +/- 5.64 | 1.66 +/- 0.15 | 7.35 +/- 0.97 | 1.83 +/- 0.14 |
Number of Infractions | Training Conditions | New Town | New Weather | New Town & Weather |
---|---|---|---|---|
Opposite Lane | 0.0 +/- 0.0 | 3.0 +/- 0.0 | 0.0 +/- 0.0 | 3.33 +/- 0.47 |
Sidewalk | 2.33 +/- 0.47 | 3.33 +/- 0.47 | 0.0 +/- 0.0 | 5.67 +/- 0.47 |
Collision-static | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 |
Collision-car | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 |
Collision-pedestrian | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 |
Number of Infractions | Training Conditions | New Town | New Weather | New Town & Weather |
---|---|---|---|---|
Opposite Lane | 2.67 +/- 0.47 | 17.33 +/- 1.7 | 0.33 +/- 0.47 | 20.0 +/- 1.41 |
Sidewalk | 3.0 +/- 0.0 | 19.67 +/- 1.25 | 0.33 +/- 0.47 | 28.33 +/- 0.94 |
Collision-static | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 |
Collision-car | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 |
Collision-pedestrian | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 |
Number of Infractions | Training Conditions | New Town | New Weather | New Town & Weather |
---|---|---|---|---|
Opposite Lane | 4.0 +/- 0.82 | 11.0 +/- 0.0 | 1.67 +/- 0.47 | 13.0 +/- 0.82 |
Sidewalk | 4.33 +/- 0.94 | 40.33 +/- 1.25 | 3.0 +/- 0.82 | 59.33 +/- 1.7 |
Collision-static | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 |
Collision-car | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 |
Collision-pedestrian | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 | 0.0 +/- 0.0 |
Number of Infractions | Training Conditions | New Town | New Weather | New Town & Weather |
---|---|---|---|---|
Opposite Lane | 3.33 +/- 0.47 | 16.67 +/- 1.89 | 2.33 +/- 1.89 | 22.67 +/- 0.47 |
Sidewalk | 5.33 +/- 1.25 | 38.33 +/- 3.4 | 8.33 +/- 1.89 | 56.33 +/- 6.85 |
Collision-static | 15.33 +/- 3.09 | 67.33 +/- 10.4 | 17.0 +/- 4.97 | 94.0 +/- 13.49 |
Collision-car | 56.0 +/- 6.38 | 90.33 +/- 11.67 | 24.0 +/- 5.72 | 139.67 +/- 8.73 |
Collision-pedestrian | 5.33 +/- 1.7 | 15.0 +/- 1.41 | 4.67 +/- 0.47 | 20.67 +/- 1.7 |