Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity

This repository is an PyTorch implementation for to paper "Hyperparameter-Free Out-of-Distribution Detection Using Softmax of Scaled Cosine Similarity" [arxiv] by Engkarat Techapanurak and Takayuki Okatani. Influenced by metrics learning, our classifier is built utilizing the cosine similarity with scaling factor. It allows our network to improve ability on out-of-distribution detection. The results from paper are shown below.

If you find this work useful in your research, please cite:

@article{techapanurak2019cosine,
  title={Hyperparameter-free out-of-Distribution detection using softmax of scaled cosine similarity},
  author={Techapanurak, Engkarat and Okatani, Takayuki},
  journal={arXiv:1905.10628},
  year={2019}
}

Datasets and Checkpoints

Datasets and checkpoints used in this implementation can be downloaded at datasets and checkpoints. For datasets, please extract and store them below 'data' directory such that './data/ood_dataset/' exists. For checkpoint, please put them below 'ckpt' directory such that './ckpt/std/' and './ckpt/cos/' exist.

Requirements

Please install required packages, using this command:

pip install -r requirements.txt

Other requirements:

• GPU: 1 unit
• Python version: 3.7

Running the code

1. Using pre-trained checkpoints

To examine the out-of-distribution detection result using pre-trained network, checkpoints should be stored as described above. Then, run commands:

Standard Network

# Standard DenseNet with CIFAR-10
python std_ood_detector.py --nn dense-100 --tr_dset cifar10

# Standard DenseNet with CIFAR-100
python std_ood_detector.py --nn dense-100 --tr_dset cifar100

# Standard WRN-28-10 with CIFAR-10
python std_ood_detector.py --nn wrn-28-10 --tr_dset cifar10

# Standard WRN-28-10 with CIFAR-100
python std_ood_detector.py --nn wrn-28-10 --tr_dset cifar100

Scaled Cosine Network

# Scaled Cosine DenseNet with CIFAR-10
python cosine_ood_detector.py --nn dense-100 --tr_dset cifar10

# Scaled Cosine DenseNet with CIFAR-100
python cosine_ood_detector.py --nn dense-100 --tr_dset cifar100

# Scaled Cosine WRN-28-10 with CIFAR-10
python cosine_ood_detector.py --nn wrn-28-10 --tr_dset cifar10

# Scaled Cosine WRN-28-10 with CIFAR-100
python cosine_ood_detector.py --nn wrn-28-10 --tr_dset cifar100

2. Training from scratch

To re-confirm the out-of-distribution detection performance, network can also be trained from scratch. To do so, one needs to remove checkpoint files of that specific task, e.g. running scaled cosine DenseNet with CIFAR-10 from scratch requires ./ckpt/cos/dense-100/cifar10/ to be empty, no any checkpoint exists in the directory.