Efficient Learning on Point Clouds with Basis Point Sets
Update: pure PyTorch implementation of the BPS encoding is now available, thanks to Omid Taheri.
Basis Point Set (BPS) is a simple and efficient method for encoding 3D point clouds into fixed-length representations.
It is based on a simple idea: select k fixed points in space and compute vectors from these basis points to the nearest points in a point cloud; use these vectors (or simply their norms) as features:
The basis points are kept fixed for all the point clouds in the dataset, providing a fixed representation of every point cloud as a vector. This representation can then be used as input to arbitrary machine learning methods, in particular it can be used as input to off-the-shelf neural networks.
Below is the example of a simple model using BPS features as input for the task of mesh registration over a noisy scan:
FAQ: what are the key differences between standard occupancy voxels, TSDF and the proposed BPS representation?
Check our ICCV 2019 paper for more details.
If you find our work useful in your research, please consider citing:
@inproceedings{prokudin2019efficient,
title={Efficient Learning on Point Clouds With Basis Point Sets},
author={Prokudin, Sergey and Lassner, Christoph and Romero, Javier},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
pages={4332--4341},
year={2019}
}
pip3 install git+https://github.com/sergeyprokudin/bps
Converting point clouds to BPS representation takes few lines of code:
import numpy as np
from bps import bps
# batch of 100 point clouds to convert
x = np.random.normal(size=[100, 2048, 3])
# optional point cloud normalization to fit a unit sphere
x_norm = bps.normalize(x)
# option 1: encode with 1024 random basis and distances as features
x_bps_random = bps.encode(x_norm, bps_arrangement='random', n_bps_points=1024, bps_cell_type='dists')
# option 2: encode with 32^3 grid basis and full vectors to nearest points as features
x_bps_grid = bps.encode(x_norm, bps_arrangement='grid', n_bps_points=32**3, bps_cell_type='deltas')
# the following tensor can be provided as input to any Conv3D network:
x_bps_grid = x_bps_grid.reshape([-1, 32, 32, 32, 3])
Clone the repository and install the dependencies:
git clone https://github.com/sergeyprokudin/bps
cd bps
python setup.py install
pip3 install torch h5py
Check one of the provided examples:
python bps_demos/train_modelnet_mlp.py
python bps_demos/train_modelnet_conv3d.py
You can directly download the results (predicted alignments, computed correspondences, demo video) here.
Results are also visualised in this video.
mkdir data
cd data
wget --output-document=mesh_regressor.h5 https://www.dropbox.com/s/u3d1uighrtcprh2/mesh_regressor.h5?dl=0
Run the model, providing the paths to your own *.ply file and output directory. You can test that everything works by running the following synthetic example:
cd bps_demos
python run_alignment.py demo_scan.ply ../logs/demo_output
If a directory is provided as a first parameter, the alignment model will be ran on all *.ply files found.
Here is an example of a prediction on some noisy real scan:
This library is licensed under the MIT-0 License. See the LICENSE file.
Note: this is the official fork of the Amazon repository written by the same author during the time of internship. Latest features and bug fixes are likely to appear here first.