One Shot Face Swapping On Megapixels Save

Abstract

Face swapping has both positive applications such as entertainment, human-computer interaction, etc. and negative applications such as DeepFake threats to politics, economics, etc. Nevertheless, it is necessary to understand the scheme of advanced methods for high-quality face swapping and generate enough and representative face swapping images to train DeepFake detection algorithms. This paper proposes the first Megapixel level method for one shot Face Swapping (or MegaFS for short). Firstly, MegaFS organizes face representation hierarchically by the proposed Hierarchical Representation Face Encoder (HieRFE) in an extended latent space to maintain more facial details, rather than compressed representation in previous face swapping methods. Secondly, a carefully designed Face Transfer Module (FTM) is proposed to transfer the identity from a source image to the target by a non-linear trajectory without explicit feature disentanglement. Finally, the swapped faces can be synthesized by StyleGAN2 with the benefits of its training stability and powerful generative capability. Each part of MegaFS can be trained separately so the requirement of our model for GPU memory can be satisfied for megapixel face swapping. In summary, complete face representation, stable training, and limited memory usage are the three novel contributions to the success of our method. Extensive experiments demonstrate the superiority of MegaFS and the first megapixel level face swapping database is released for research on DeepFake detection and face image editing in the public domain.

How to Use

Please refer inference README.md for more details.

MegaFS Dataset

This dataset is built for forgery detection and face swapping research, as test dataset and comparative results respectively.

Based on CelebA-HQ, we have swapped faces on and divided them into three subsets according to the three swapping modules designed in our paper.

Following the license of CelebA-HQ, the dataset is made freely available for non-commercial use under the Creative Commons CC BY-NC 4.0 license.

Each image is named as targetID_sourceID.jpg, where ID represents the image offset in CelebA-HQ, source image provides the identity and target image provides other information.

Dataset Download Links

The data includes swapped faces on CelebA-HQ by MegaFS(FTM, ID injection, LCR).

In addition, MegaFS-FF++.zip contains the swapped faces in frames (by MegaFS 256px version) and the 889 ID folders for future research.

• Google Drive
• Baidu Cloud (access code: 6bbh)

Citation

You can find the paper at this link.

If you found this dataset is helpful, please cite:

@inproceedings{zhu2021megafs,
  title={One Shot Face Swapping on Megapixels},
  author={Zhu, Yuhao and Li, Qi and Wang, Jian and Xu, Chengzhong and Sun, Zhenan},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)},
  pages = {4834-4844},
  month = {June},
  year={2021}
}