(ECCV 2020) RANSAC-Flow: generic two-stage image alignment
Pytorch implementation of paper "RANSAC-Flow: generic two-stage image alignment" (ECCV 2020)
[PDF] [Project page] [Demo] [Youtube demo]
If our project is helpful for your research, please consider citing :
@inproceedings{shen2020ransac,
title={RANSAC-Flow: generic two-stage image alignment},
author={Shen, Xi and Darmon, Fran{\c{c}}ois and Efros, Alexei A and Aubry, Mathieu},
booktitle={16th European Conference on Computer Vision}
year={2020}
}
Since some functions have different behaviors in different Pytorch version, we recommand to install EXACT version indicated in the Dependencies if you want to reproduce our results in the paper. For more details, please refer to this issue.
Input | Our Fine Alignment | ||
---|---|---|---|
Animation | Avg | Animation | Avg |
Source | Target | 3D Reconstruction |
---|---|---|
Source | Target | Texture Transfer |
---|---|---|
Other results (such as: aligning duplicated artworks, optical flow, localization etc.) can be seen in our paper.
If you have old nvidia GPU which has the compute capability < 8.0, set an environment variable as;
export CUDA_VERSION=10
Otherwise, you don't need to do anything.
Install Nvidia driver and run below:
./script/install_configure_nvidia_container_toolkit.sh
docker compose build
docker compose run --rm app bash
run "3. Quick Start" part by running below command inside the docker.
cd quick_start
./runjupyter_fromdocker.sh
Then open "localhost:8888" in your host browser. You can also run on your own image by 3.2 way.
Our model can be learnt in a single GPU GeForce GTX 1080Ti (12G).
Install Pytorch adapted to your CUDA version :
Other dependencies (tqdm, visdom, pandas, kornia, opencv-python) :
pip install -r requirements_cuda10.txt
Quick download :
./model/pretrained/download_model.sh
For more details of the pre-trained models, see here
Download the results of ArtMiner :
./data/Brueghel_detail.sh # Brueghel detail dataset (208M) : visual results, aligning groups of details
Download our training data here (~9G). It includes the validation and test data as well.
A quick start guide of how to use our code is available in demo.ipynb
You can run RANSAC-FLOW as below
cd quick_start
python align2images.py --img1='../img/ArtMiner_Detail_Res13_10.png' --img2='../img/ArtMiner_Detail_Res13_11.png'
Then, you can find the output images in output/ directory.
To run the training, we need pairs that are coarsely aligned. We provide a notebook to show how to generate the training pairs. Note that, we also provide our training pairs in here.
The training data need to be downloaded from here and saved into ./data
. The file structure is :
./RANSAC-Flow/data/MegaDepth
├── MegaDepth_Train/
├── MegaDepth_Train_Org/
├── Val/
└── Test/
As mentioned in the paper, the model trained on MegaDepth contains the following 3 different stages of training:
cd train/
bash stage1.sh
cd train/
bash stage2.sh
cd train/
bash stage3.sh
If you want to conduct fine-tuning on your own dataset. It is recommended to start from our MegaDepth trained model. You can see all the arguments of training by :
cd train/
python train.py --help
If you don't need to predict the matchability, you can set the weight of the matchability loss to 0 (--eta 0 in the train.py), and set your path of images (--trainImgDir). Please refer to train/stage2.sh for other arguments.
In case of predicting matchability, you need to tune the weight of the matchability loss (argument --eta in the train.py) depending on the dataset.
The evaluation of different tasks can be seen in the following files:
We appreciate helps from :
Jiahui Zhang pointed a bug for YFCC evaluation, see here for the udpate
Part of code is borrowed from our previous projects: ArtMiner and Watermark
Remove useless parts + rename some functions / parameters to make it compatible with papers + more comments
Fix bug in YFCC evaluation, see here. results in the paper have been updated as well.
Make a comparison to recent work GLU-Net, results are updated in the paper.
Add csv file containing annotated coorespondences for RobotCar, see here for more details.