IS, FID score Pytorch and TF implementation, TF implementation is a wrapper of the official ones.
This repo contains information/implementation (PyTorch, Tensorflow) about IS and FID score. This is a handy toolbox that you can easily add to your projects. TF implementations are intended to compute the exact same output as the official ones for reporting in papers. Discussion/PR/Issues are very welcomed.
Put this metrics/
folder in your projects, and see below (Pytorch), and each .py's head comment for usage.
We also need to download some files in res/, see res/README.md for more details.
Requirements
mean=9.67278, std=0.14992
for CIFAR-10 train data when n_split=10nn.DataParallel
CUDA_VISIBLE_DEVICES=0,1,2,3
will use 4 GPU.command line usage
calculate IS, FID
# calc IS score on CIFAR10, will download CIFAR10 data to ../data/cifar10
python is_fid_pytorch.py
# calc IS score on custom images in a folder/
python is_fid_pytorch.py --path foldername/
# calc IS, FID score on custom images in a folder/, compared to CIFAR10 (given precalculated stats)
python is_fid_pytorch.py --path foldername/ --fid res/stats_pytorch/fid_stats_cifar10_train.npz
# calc FID on custom images in two folders/
python is_fid_pytorch.py --path foldername1/ --fid foldername2/
# calc FID on two precalculated stats
python is_fid_pytorch.py --path res/stats_pytorch/fid_stats_cifar10_train.npz --fid res/stats_pytorch/fid_stats_cifar10_train.npz
precalculate stats
# precalculate stats store as npz for CIFAR 10, will download CIFAR10 data to ../data/cifar10
python is_fid_pytorch.py --save-stats-path res/stats_pytorch/fid_stats_cifar10_train.npz
# precalculate stats store as npz for images in folder/
python is_fid_pytorch.py --path foldername/ --save-stats-path res/stats_pytorch/fid_stats_folder.npz
in code usage
mode=1
: image tensor has already normalized by mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]
mode=2
: image tensor has already normalized by mean=[0.500, 0.500, 0.500], std=[0.500, 0.500, 0.500]
from metrics import is_fid_pytorch
# using precalculated stats (.npz) for FID calculation
is_fid_model = is_fid_pytorch.ScoreModel(mode=2, stats_file='res/stats_pytorch/fid_stats_cifar10_train.npz', cuda=cuda)
imgs_nchw = torch.Tensor(50000, C, H, W) # torch.Tensor in -1~1, normalized by mean=[0.500, 0.500, 0.500], std=[0.500, 0.500, 0.500]
is_mean, is_std, fid = is_fid_model.get_score_image_tensor(imgs_nchw)
# we can also pass in mu, sigma for get_score_image_tensor()
is_fid_model = is_fid_pytorch.ScoreModel(mode=2, cuda=cuda)
mu, sigma = is_fid_pytorch.read_stats_file('res/stats_pytorch/fid_stats_cifar10_train.npz')
is_mean, is_std, fid = is_fid_model.get_score_image_tensor(imgs_nchw, mu1=mu, sigma1=sigma)
# if no need FID
is_fid_model = is_fid_pytorch.ScoreModel(mode=2, cuda=cuda)
is_mean, is_std, _ = is_fid_model.get_score_image_tensor(imgs_nchw)
# if want stats (mu, sigma) for imgs_nchw, send in return_stats=True
is_mean, is_std, _, mu, sigma = is_fid_model.get_score_image_tensor(imgs_nchw, return_stats=True)
# from pytorch dataset, use get_score_dataset(), instead of get_score_image_tensor(), other usage is the same
cifar = dset.CIFAR10(root='../data/cifar10', download=True,
transform=transforms.Compose([
transforms.Resize(32),
transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
)
IgnoreLabelDataset(cifar)
is_mean, is_std, _ = is_fid_model.get_score_dataset(IgnoreLabelDataset(cifar))
Assumption
Formulation
Explanation
Reference
Formulation
Reference