DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates
[project] [paper] [demo] [animations]
DeepMetaHandles is a shape deformation technique. It learns a set of meta-handles for each given shape. The disentangled meta-handles factorize all the plausible deformations of the shape, while each of them corresponds to an intuitive deformation direction. A new deformation can then be generated by the "linear combination" of the meta-handles. Although the approach is learned in an unsupervised manner, the learned meta-handles possess strong interpretability and consistency.
conda env create -f environment.yml
.data/demo
and checkpoints/chair_15.pth
from here and place them in the corresponding folder. Pre-processed demo data contains the manifold mesh, sampled control point, sampled surface point cloud, and corresponding biharmonic coordinates.src/demo_target_driven_deform.py
to deform a source shape to match a target shape.src/demo_meta_handle.py
to generate deformations along the direction of each learned meta-handle.data/chair
from here and place them in the corresponding folder.src/train.py
to start training.Note: For different categories, you may need to adjust the number of meta-handles. Also, you need to tune the weights for the loss functions. Different sets of weights may produce significantly different results.
data_preprocessing/.
data_preprocessing/normalize_bin
to normalize the manifold into a unit bounding sphere.--output xxx.mesh
option to generate the .mesh
format tet mesh. Also, you will get a xxx.mesh__sf.obj
for the surface mesh. We will use xxx.mesh
and xxx.mesh__sf.obj
to calculate the biharmonic weights. We will only deform xxx.mesh__sf.obj
later.data_preprocessing/sample_key_points_bin
to sample control points from xxx.mesh__sf.obj
. We use the FPS algorithm over edge distances to sample the control points.data_preprocessing/calc_weight_bin
to calculate the bihrnomic weights. It takes xxx.mesh
, xxx.mesh__sf.obj
, and the control point file as input, and will output a text file containing the weight matrix for the vertices in xxx.mesh__sf.obj
.data_preprocessing/sample_surface_points_bin
to sample points on the xxx.mesh__sf.obj
and calculate the corresponding biharmonic weights for the sampled point cloud.data_preprocessing/merge_data
to convert cpp output into numpy files.If you find our work useful, please consider citing our paper:
@article{liu2021deepmetahandles,
title={DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates},
author={Liu, Minghua and Sung, Minhyuk and Mech, Radomir and Su, Hao},
journal={arXiv preprint arXiv:2102.09105},
year={2021}
}