Single Image to 3D Textured Mesh in 10 seconds with Convolutional Reconstruction Model.
Official implementation for CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model.
CRM is a feed-forward model which can generate 3D textured mesh in 10 seconds.
https://github.com/thu-ml/CRM/assets/40787266/8b325bc0-aa74-4c26-92e8-a8f0c1079382
Install package one by one, we use python 3.9
pip install torch==1.13.0+cu117 torchvision==0.14.0+cu117 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu117
pip install torch-scatter==2.1.1 -f https://data.pyg.org/whl/torch-1.13.1+cu117.html
pip install kaolin==0.14.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-1.13.1_cu117.html
pip install -r requirements.txt
besides, one by one need to install xformers manually according to the official doc (conda no need), e.g.
pip install ninja
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers
Install nvdiffrast according to the official doc, e.g.
pip install git+https://github.com/NVlabs/nvdiffrast
We suggest gradio for a visualized inference.
gradio app.py
For inference in command lines, simply run
CUDA_VISIBLE_DEVICES="0" python run.py --inputdir "examples/kunkun.webp"
It will output the preprocessed image, generated 6-view images and CCMs and a 3D model in obj format.
Tips: (1) If the result is unsatisfatory, please check whether the input image is correctly pre-processed into a grey background. Otherwise the results will be unpredictable. (2) Different from the Huggingface Demo, this official implementation uses UV texture instead of vertex color. It has better texture than the online demo but longer generating time owing to the UV texturing.
@article{wang2024crm,
title={CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model},
author={Zhengyi Wang and Yikai Wang and Yifei Chen and Chendong Xiang and Shuo Chen and Dajiang Yu and Chongxuan Li and Hang Su and Jun Zhu},
journal={arXiv preprint arXiv:2403.05034},
year={2024}
}