A python implementation of portrait lighting transfer using a mass transport approach.
A python implementation of portrait lighting transfer using a mass transport approach.
3DMM model in original implementation is replaced by PRNet.
input img | reference img | relighting with color | with light
sh env.sh
cd python_portrait_relight/cython/
python setup.py build_ext -i
# python demo.py
# fast = False(rendering with numpy version)
portrait_s1.jpg: 800x800x3
portrait_r1.jpg: 873x799x3
render_texture: 16.000
render_texture: 13.967
relight time with color: 36.16
render_texture: 13.871
render_texture: 13.617
relight time with light: 30.65
...
# python demo.py
# fast = True(rendering with c++ version)
portrait_s1.jpg: 800x800x3
portrait_r1.jpg: 873x799x3
render_texture: 0.026
render_texture: 0.024
relight time with color: 6.27
render_texture: 0.025
render_texture: 0.024
relight time with light: 3.20
...
Let input image be I, reference image be R and output image be O.
Let posI, posR be frontal 3d face position map of img I, R, with shape=[n, 3].\
Let colorI, colorR be rgb colors of the reconstructed vertices of img I, R, with shape=[n, 3].
Let normalI, normalR be normal vectors of the vertices of img I, R, with shape=[n, 3].
We obtain features fI=[colorI, posI[:,:,:2], nomralI], fR=[colorR, posR[:,:,:2], normalR] of img I, R, with shape=[n, 8].
Then we determine pdf transfer function t, so that f{t(fI)}=f{fR}, where f{x} is the probability density function of array x.
t(colorI) is the relighted image of I with R for reference.
Finally, we apply regrain algorithm for postprocessing.
portrait lighting transfer using a mass transport approach
by Zhixin Shu, Sunil Hadap, Eli Shechtman, Kalyan Sunkavalli, Sylvain Paris and Dimitris Samaras.
Author's matlab implementation