ALL-CNN

This is the implementation of All-CNN-C model for CIFAR-10 from the paper Striving for Simplicity: The All Convolutional Net by Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, Martin Riedmiller, accepted as a workshop contribution at ICLR 2015.

The best saved model, from iteration 52000, achieves 90.25% accuracy on the test set. It was trained on Caffe version from commit 5a201dd960840c319cefd9fa9e2a40d2c76ddd73.

Dataset

The training was on global contrast normalized and ZCA whitened CIFAR-10 whithout any data augmentation. Both training and test sets were created with Pylearn2 library using make_cifar10_gcn_whitened.py script that outputs train.pkl and test.pkl files. Then transformed to LMDB database with random, unique keys (Creating an LMDB database in Python).

Here are the links to download preprocessed and ready for training/testing CIFAR-10 LMDB databases:

• cifar-10_train_lmdb.zip (SHA-1 checksum: e72b00259625f77fd6c4b952568dd0904b3fa75c)
• cifar-10_test_lmdb.zip (SHA-1 checksum: ebe7e7ccbf13d704c4ce4147bcadab9dd3511972)

Training

caffe train -solver ALL_CNN_C_solver.prototxt -gpu all |& tee ALL_CNN_C.log

I found the training unstable. You may need to run it more than once to achieve sub 10% error rate.

On NVIDIA TITAN X GPU the training took about 2.5 h.

Testing

caffe test -model ALL_CNN_C_train_val.prototxt -weights ALL_CNN_C_iter_52000.caffemodel