NVIDIA Kaolin Wisp is a PyTorch library powered by NVIDIA Kaolin Core to work with neural fields (including NeRFs, NGLOD, instant-ngp and VQAD).
BaseAS
(acceleration structure, i.e. for raymarching), WispModule
(any wisp module that can display properties in the renderer).Full Changelog: https://github.com/NVIDIAGameWorks/kaolin-wisp/compare/v0.1.1...v0.1.2
The second release of Wisp features:
Full Changelog: https://github.com/NVIDIAGameWorks/kaolin-wisp/compare/v0.1.0...v0.1.1
First kaolin-wisp release. Supports full optimization pipelines of neural radiance fields and signed distance functions. Also includes an interactive renderer to visualize the optimization progress.
An Octree allows for fast location based queries and raymarching operations. On NVIDIA RTX 3090, that brings NeRF training time down to anywhere between tens of seconds to single minutes for single objects like the bulldozer. (The exact performance is affected by other blocks in the neural pipeline: tracer, feature structure, etc..)
The initial version of wisp is shipped with 4 types of feature grids:
Support for Positional Encoding (Fourier), torch MLP Decoders.
The tracers framework supports types of channels (i.e: rgba, sdf) 2 implementations of differentiable ray tracers are included.
'voxel'
- fixed amount of samples will be generated per "octree cell"'ray'
- fixed amount of samples will be generated between the near / far planes of the camera.RenderBuffer
- smart pixel buffers used by tracers. Support custom types of Channel
s and various blending and channel normalization modes.Ray
- batched ray packs for ray generationA toolkit for loading, processing and evaluation of various modalities is included in this version (including images, meshes, sdfs and kaolin's Structured Point Clouds.
wisp ships with data loaders support for:
2 trainers are currently supported (for NeRF and SDFs). Trainers can run with or without the interactive renderer.
Wisp includes an interactive renderer with flexible support for new types of neural objects. The renderer was built in mind to support future neural pipelines which may not yet exist. Dependencies: pycuda, glumpy, glfw, imgui.
Full Changelog: https://github.com/NVIDIAGameWorks/kaolin-wisp/commits/v0.1.0