Coherent Semantic Attention for image inpainting(ICCV 2019)
Arxiv |ICCV 2019 paper| BibTex
The existing inpainting methods often generate contents with blurry textures and distorted structures due to the discontinuity of the local pixels.From a semantic-level perspective, the local pixel discontinuity is mainly because these methods ignore the semantic relevance and feature continuity of hole regions. To handle this problem, we investigate the human behavior in repairing pictures and propose a fined deep generative model-based approach with a novel coherent semantic attention (CSA) layer, which can not only preserve contextual structure but also make more effective predictions of missing parts by modeling the semantic relevance between the holes features. Meanwhile, we further propose consistency loss and feature patch discriminator to stabilize the network training process and improve the details.
Clone this repo:
git clone https://github.com/KumapowerLIU/CSA-inpainting.git
cd CSA-inpainting
Install PyTorch and dependencies from http://pytorch.org
Or you can clone the repo and run by conda
conda env create -f environment.yaml
conda activate csa-inpainting
jupyter notebook
Open the browser and enter the URL http://localhost:8080
to enjoy jupyter notebook
We use Places2, CelebA and Paris Street-View datasets. To train a model on the full dataset, download datasets from official websites.
Our model is trained on the irregular mask dataset provided by Liu et al. You can download publically available Irregular Mask Dataset from their website.
CC 4.0 Attribution-NonCommercial International. The software is for educaitonal and academic research purpose only.
@InProceedings{Liu_2019_CSA,
Author = {Hongyu Liu and Bin Jiang and Yi Xiao and Chao Yang},
Title = {Coherent Semantic Attention for Image Inpainting},
booktitle = { IEEE International Conference on Computer Vision (ICCV)},
month = {July},
year = {2019}
}
We benefit a lot from Shift-net