Code for CVPR 2019 paper "ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging"
This repository contains code to analyze and predict contact maps for human grasping, presented in the paper
ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging - Samarth Brahmbhatt, Cusuh Ham, Charles C. Kemp, and James Hays, CVPR 2019
Paper (CVPR 2019 Oral) | Explore the dataset | Poster | Slides
Please see contactdb_utils for access to raw ContactDB data, and code to process it; analysis branch for code to generate various analysis graphs from the paper.
git clone https://github.com/samarth-robo/contactdb_prediction.git
. Commands for the following steps should be executed from the contactdb_prediction
directory.contactdb_prediction
environment: conda create env -f environment.yml
, and activate it: source activate contactdb_prediction
.CONTACTDB_DATA_DIR
, make a symlink to it: ln -s CONTACTDB_DATA_DIR data/voxelized_meshes
.CONTACTDB_MODELS_DIR
, make a symlink to it: ln -s CONTACTDB_MODELS_DIR data/checkpoints
.We propose two methods to make diverse contact map predictions: DiverseNet and Stochastic Multiple Choice Learning (sMCL). This branch has code for the diversenet models. Checkout the smcl branch for sMCL code.
Predict contact maps for the 'use' instruction, using the voxel grid 3D representation:
$ python eval.py --instruction use --config configs/voxnet.ini --checkpoint data/checkpoints/use_voxnet_diversenet_release/checkpoint_model_86_val_loss\=0.01107167.pth
pan error = 0.0512
mug error = 0.0706
wine_glass error = 0.1398
You can add the --show object <pan | mug | wine_glass>
flag to show the 10 diverse predictions:
$ python eval.py --instruction use --config configs/voxnet.ini --checkpoint data/checkpoints/use_voxnet_diversenet_release/checkpoint_model_86_val_loss\=0.01107167.pth --show_object mug
mug error = 0.0706
In general, the command is
python eval.py --instruction <use | handoff> --config <configs/voxnet.ini | configs/pointnet.ini> --checkpoint <checkpoint filename>
Use the following checkpoints:
Method | Checkpoint |
---|---|
Use - VoxNet | data/checkpoints/use_voxnet_diversenet_release/checkpoint_model_86_val_loss=0.01107167.pth |
Use - PointNet | data/checkpoints/use_pointnet_diversenet_release/checkpoint_model_29_val_loss=0.6979221.pth |
Handoff - VoxNet | data/checkpoints/handoff_voxnet_diversenet_release/checkpoint_model_167_val_loss=0.01268427.pth |
Handoff - PointNet | data/checkpoints/handoff_pointnet_diversenet_release/checkpoint_model_745_val_loss=0.5969936.pth |
Start the visdom
server
$ source activate contactdb_prediction
$ visdom
The base training command is
python train_val.py --instruction <use | handoff> --config <configs/voxnet.ini | configs/pointnet.ini> [--device <GPU ID> --checkpoint_dir <directory where checkpints are saved> --data_dir <directory where data is downloaded>]
@inproceedings{brahmbhatt2018contactdb,
title={{ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging}},
author={Samarth Brahmbhatt and Cusuh Ham and Charles C. Kemp and James Hays},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2019},
note={\url{https://contactdb.cc.gatech.edu}}
}