Caricature Your Face

💡 Using The Face of Arts and DST to generate geometry-aware face caricatures.

📖 This project in details with analysis on Medium: Geometry-Aware Style Transfer: Implementation and Analysis

Requirements

• Python 3
• pytorch, torchvision, cudatoolkit, numpy, PIL, matplotlib, sklearn, cv2, imutils, scikit-image

Set up the virtual environment:

python3 -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt

after setup, you can just run source venv/bin/activate to activate the venv later on.

How To Run

preprocessing.sh is used to convert all Face of Art style images' correspondence points into .txt, this only need to be run once if you don't already have data/style/pts containing all Face of Art style images.

run.sh is a script automating the two steps for geometry style transfer.

Currently, run.sh serve as a demo with Barack Obama as the content image and Vincent van Gogh's self-portrait with the style image.

1. Obtain Facial Landmark

This step will obtain the facial landmarks for the content image with dlib.

python face_landmark_detection.py ${content_path} ${style_path} ${content_pts_path} ${style_pts_path_dlib}

2. Run Style Transfer

Now, run deformable style transfer, this step will take ~1.5 minutes on a single GPU.

Please check parameter adjustment in DST repo [code].

python main.py ${content_path} ${style_path} ${content_pts_path} ${style_pts_path} \
  ${output_dir} ${output_prefix} ${im_size} ${max_iter} \
  ${checkpoint_iter} ${content_weight} ${warp_weight} ${reg_weight} ${optim} \
  ${lr} ${verbose} ${save_intermediate} ${save_extra} ${device}

Results

Demo

Result from run.sh with Barack Obama as the content image and Vincent van Gogh's self-portrait with the style image. Images from left to right are: input content image, input style image, a standard style transfer output, demo output.

More Results

Acknowledgment

• Dilb facial landmark
• Deformable Style Transfer (DST) [code] [paper]
• The Face of Art: Landmark Detection and Geometric Style in Portraits [website] [paper]
• Style Transfer by Relaxed Optimal Transport and Self-Similarity. Nicholas Kolkin, Jason Salavon and Gregory Shakhnarovich. CVPR 2019. [paper] [code] [David Futschik's implementation]
• WarpGAN: Automatic Caricature Generation. Yichun Shi, Debayan Deb and Anil K. Jain. CVPR 2019. [paper] [code]
• Neural Best-Buddies: Sparse Cross-Domain Correspondence. Kfir Aberman, Jing Liao, Mingyi Shi, Dani Lischinski, Baoquan Chen and Daniel Cohen-Or. SIGGRAPH 2018. [paper] [code]