SMOL-V: like Vulkan/Khronos SPIR-V, but smaller.
SMOL-V encodes Vulkan/Khronos SPIR-V format programs into a form that is smoller, and is more compressible. Normally no changes to the programs are done; they decode into exactly same program as what was encoded. Optionally, debug information can be removed too.
SPIR-V is a very verbose format, several times larger than same programs expressed in other shader formats (e.g. DX11 bytecode, GLSL, DX9 bytecode etc.). The SSA-form with ever increasing IDs is not very appreciated by regular data compressors either. SMOL-V does several things to improve this:
A somewhat similar utility is spirv-remap from glslang.
See this blog post for more information about how I did SMOL-V.
Add source/smolv.h
and source/smolv.cpp
to your C++ project build.
It might require C++11 or somesuch. There are
Github Actions set up for this project, that build on Windows (VS2017),
macOS (Xcode 11.1) and Linux (Ubuntu 16 / gcc 5.4).
smolv::Encode
and smolv::Decode
is the basic functionality. See smolv.h.
Other functions are for development/statistics purposes, to figure out frequencies and distributions of the instructions.
There's a test + compression benchmarking suite in testing/testmain.cpp
, using that needs adding
other files under testing/external to the build too (3rd party code: glslang remapper, Zstd, LZ4, miniz).
See Changelog.
Code itself: Public Domain or MIT, pick whichever works better for you.
There is 3rd party code under the testing framework (testing/external
); it is not required for
using SMOL-V. Most of that code (glslang,
LZ4, Zstd, sokol_time.h)
is BSD or zlib-licensed, and taken from github repositories of the respective projects. miniz
is public domain.
There are SPIR-V binary shader dumps under tests/spirv-dumps
for compression testing;
these are not required for using SMOL-V. Not sure how to appropriately
"license" them (but hey they are kinda useless by themselves out of context),
so I'll go with this: "Binary shader dumps under 'tests' folder are only to be
used for SMOL-V testing". Details on them:
tests/spirv-dumps/dota2
- some shaders from DOTA2, Copyright Valve Corporation, all rights reserved.tests/spirv-dumps/shadertoy
- several most popular shaders from Shadertoy, converted to Vulkan
SPIR-V via glslang. Copyrights by their individual authors (filename matches last component of shadertoy URL).tests/spirv-dumps/talos
- some shaders from The Talos Principle,
Copyright (c) 2002-2016 Croteam All rights reserved.tests/spirv-dumps/unity
- various Unity shaders, produced
through a HLSL -> DX11 bytecode -> HLSLcc -> glslang toolchain.As of 2020 May 25 29, results on 372 shaders (under tests/spirv-dumps
) are:
Compressed with <none>:
Raw 5188.9KB 100.0%
Remapper 5089.0KB 98.1%
SmolV 1934.2KB 37.3%
Compressed with zlib:
Raw 1301.3KB 25.1%
Remapper 1230.5KB 23.7%
SmolV 696.6KB 13.4%
Compressed with LZ4 HC:
Raw 1448.7KB 27.9%
Remapper 1303.9KB 25.1%
SmolV 711.3KB 13.7%
Compressed with Zstandard:
Raw 983.4KB 19.0%
Remapper 870.8KB 16.8%
SmolV 541.3KB 10.4%
Compressed with Zstandard 20:
Raw 649.6KB 12.5%
Remapper 599.1KB 11.5%
SmolV 419.7KB 8.1%
Decoding these 372 shaders from SMOL-V back into SPIR-V takes 10.2ms (VS2017, x64 Release, AMD ThreadRipper 1950X 3.4GHz, one thread).