Source code for CVPR 2020 paper "Scene-Adaptive Video Frame Interpolation via Meta-Learning"
Source code for CVPR 2020 paper "Scene-Adaptive Video Frame Interpolation via Meta-Learning"
Project | Paper-CVF | Paper-ArXiv | Supp
conda install cupy -c conda-forge
)For [DAIN], the environment is different; please check dain/dain_env.yml
for the requirements.
Disclaimer : This code is re-organized to run multiple different models in this single codebase. Due to a lot of version and env changes, the numbers obtained from this code may be different (usually better) from those reported in the paper. The original code modifies the main training scripts for each frame interpolation github repo ([DVF (voxelflow)], [SuperSloMo], [SepConv], [DAIN]), and are put in ./legacy/*.py
. If you want to exactly reproduce the numbers reported in our paper, please contact @myungsub for legacy experimental settings.
data/
folder:
ln -s /path/to/vimeo_septuplet_data/ ./data/vimeo_septuplet
other-color-allframes.zip
and other-gt-interp.zip
./pretrained_models/*.pth
./scripts/run_{VFI_MODEL_NAME}.sh
sepconv
, voxelflow
, superslomo
, cain
, and rrin
--mode val
and --pretrained_model {MODEL_NAME}
scripts/run_test.sh
for details:--data_root
to your desired dir/--img_fmt
(defaults to png
)--model
, --loss
, and --pretrained_models
to what you want
--model
should be sepconv
, and --loss
should be 1*L1
--model
should be voxelflow
, and --loss
should be 1*MSE
--model
should be superslomo
, --loss
should be 1*Super
--model
should be dain
, and --loss
should be 1*L1
--model
should be cain
, and --loss
should be 1*L1
--model
should be rrin
, and --loss
should be 1*L1
config.py
)--attenuate
in scripts/run_{VFI_MODEL_NAME}.sh
--metasgd
(This usually results in the best performance!)If you find this code useful for your research, please consider citing the following paper:
@inproceedings{choi2020meta,
author = {Choi, Myungsub and Choi, Janghoon and Baik, Sungyong and Kim, Tae Hyun and Lee, Kyoung Mu},
title = {Scene-Adaptive Video Frame Interpolation via Meta-Learning},
booktitle = {CVPR},
year = {2020}
}
The main structure of this code is based on MAML++. Training scripts for each of the frame interpolation method is adopted from: [DVF], [SuperSloMo], [SepConv], [DAIN], [CAIN], [RRIN]. We thank the authors for sharing the codes for their great works.