GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering [CVPR 2024]

webpage | arXiv | Video

Abdullah Hamdi ¹, Luke Melas-Kyriazi ¹, Guocheng Qian ^2,4, Jinjie Mai ², Ruoshi Liu ³, Carl Vondrick ³, Bernard Ghanem ², Andrea Vedaldi ¹

¹ Visual Geometry Group, University of Oxford ² KAUST, ³ Columbia University, ⁴ Snap Inc.,

Overview

We provide a PyTorch implementation of our Generalized Exponential Splatting (GES) method, as well as the Gaussian Splatting method for comparison. We also provide the code to reproduce the results in our paper. The code is heavily based on 3D Gaussian Splatting for Real-Time Radiance Field Rendering. The .ply files output of the GES are exactly the same type as the Gaussian Splatting .ply files. You can use the same rendering code and utilities to visualize the results or edit them. The extra parameter shape is fused with the other gaussian parameters before saving the .ply file.

Colab Notebook

A great way to get started with GES is to use the online notebook produced by camenduru with no requirments.

Hardware Requirements

• CUDA-ready GPU with Compute Capability 7.0+
• 24 GB VRAM (to train to paper evaluation quality)

Software Requirements

• Conda (recommended for easy setup)
• C++ Compiler for PyTorch extensions (we used Visual Studio 2019 for Windows)
• CUDA SDK 11 for PyTorch extensions, install after Visual Studio (we used 11.8, known issues with 11.6)
• C++ Compiler and CUDA SDK must be compatible

Setup

Our default, provided install method is based on Conda package and environment management:

SET DISTUTILS_USE_SDK=1 # Windows only
conda env create --file environment.yml
conda activate ges

Please note that this process assumes that you have CUDA SDK 11 installed, not 12. For modifications, see below. You can also use the same conda environment of the Gaussian Splatting repo.

Running

Download the datasets from the original repository, and place them in the tandt_db and nerf_360 directories.

For example, let's assume you download tandt_db dataset, please run following to reproduce Gaussian splatting results, but under the setup of GES.

python train_gaussian.py -s ./tandt_db/tandt/train -m ./outputs/train --eval 

And run following to reproduce our main GES results with default settings.

python train_ges.py -s ./tandt_db/tandt/train -m ./outputs/train --eval 

To reproduce all the results in our paper, prepare your datasets according to the script, and then run it:

bash ges_full_eval.sh # for our GES implementation

The detailed numerical results of GES (per scene) can be found in here

Numerical Simulation for Generlized Exponential Function (GEF)

Check the notebook simulation.ipynb here for the numerical simulation of the Generalized Exponential Function (GEF), that GES is based on, as well as additional visualizations from the paper.

Cite

If you find our work useful in your research, please consider citing:

@InProceedings{hamdi_2024_CVPR,
    author    = {Hamdi, Abdullah and Melas-Kyriazi, Luke and Mai, Jinjie and Qian, Guocheng and Liu, Ruoshi and Vondrick, Carl and Ghanem, Bernard and Vedaldi, Andrea},
    title     = {GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    year      = {2024},
}