Evaluate three types of task shifting with popular continual learning algorithms.
Evaluate three types of task shifting with popular continual learning algorithms.
This repository implemented and modularized following algorithms with PyTorch:
All the above algorithms are compared to following baselines with the same static memory overhead:
Key tables:
If this repository helps your work, please cite:
@inproceedings{Hsu18_EvalCL,
title={Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines},
author={Yen-Chang Hsu and Yen-Cheng Liu and Anita Ramasamy and Zsolt Kira},
booktitle={NeurIPS Continual learning Workshop },
year={2018},
url={https://arxiv.org/abs/1810.12488}
}
This repository was tested with Python 3.6 and PyTorch 1.0.1.post2. Part of the cases is tested with PyTorch 1.5.1 and gives the same results.
pip install -r requirements.txt
The scripts for reproducing the results of this paper are under the scripts folder.
./scripts/split_MNIST_incremental_domain.sh 0
# The last number is gpuid
# Outputs will be saved in ./outputs
===Summary of experiment repeats: 3 / 3 ===
The regularization coefficient: 400.0
The last avg acc of all repeats: [90.517 90.648 91.069]
mean: 90.74466666666666 std: 0.23549144829955856
reg_coef: 0.1 mean: 76.08566666666667 std: 1.097717733400629
reg_coef: 1.0 mean: 77.59100000000001 std: 2.100847606721314
reg_coef: 10.0 mean: 84.33933333333334 std: 0.3592671553160509
reg_coef: 100.0 mean: 90.83800000000001 std: 0.6913701372395712
reg_coef: 1000.0 mean: 87.48566666666666 std: 0.5440161353816179
reg_coef: 5000.0 mean: 68.99133333333333 std: 1.6824762174313899
Lookup available options:
python iBatchLearn.py -h
Below are CIFAR100 results. Please refer to the scripts for details.