Turbo Run Length Encoding Save

TurboRLE: Turbo Run Length Encoding

Efficient and fastest Run Length Encoding library

• ARM NEON support

• 100% C (C++ compatible headers), without inline assembly

• Most efficient compression

• No other RLE compress or decompress faster with better compression

• :sparkles: faster compression and 2x faster decompression with :+1: SIMD

• :+1: Java Critical Natives Interface. Access TurboRLE incl. SIMD! from Java

• Compress better and up to 12 times faster and decompress up to 6 times faster than other fast RLEs

• Can be faster than memcpy!

• :+1: ZERO! byte overhead

• No modification of the raw data, preserving compressibility for further postprocessing (ex. entropy coding)

• Order preserving

TurboRLE

• TRLE: TurboRLE - Efficient and fast Run Length Encoding
• SRLE: TurboRLE Escape - Fast Run Length Encoding with automatic escape determination

Benchmark:

• Single thread with TurboBench
• Realistic and practical benchmark with large files and different distributions

External functions benchmarked

• MRLE: Mespotine RLE: Run Length Encoding
• RLE8: A fast 8 bit Run Length Encoding (SSE/AVX2)

CPU: Skylake i7-6700 3.4GHz, gcc 8.3 (2019-08)

• BMP File: girl.bmp in RLE64Samples

(bold = pareto) MB=1.000.000

• Checkers program "End Game Table Base": 1034.db

• Text File: enwik9bwt enwik9 bwt generated w. libdivsufsort

CPU: ARM A73-ODROID-N2 1.8GHz, gcc 8.3 (2019-08)

• BMP File: girl.bmp in RLE64Samples

• Text File: enwik9bwt enwik9 bwt generated w. libdivsufsort

• Post-processing: Entropy Coding after Run Length Encoding
  Direct entropy encoding after "trle" (no additional "move to front" or other transformation)

for more info, see also: Entropy Coding Benchmark

Compile:

	git clone https://github.com/powturbo/Turbo-Run-Length-Encoding.git
    cd Turbo-Run-Length-Encoding

Linux + Windows MingW

	make
    or
	make AVX2=1

Windows Visual C++

	nmake /f makefile.vs
    or
	nmake AVX2=1 /f makefile.vs

Testing Run Length Encoding libraries:

    ./trle file
    ./trle -e# file

	# = function id (see file trle.c)

Environment:

OS/Compiler (32 + 64 bits):

• Windows: MinGW-w64 makefile
• Windows: Visual c++ (>=VS2008) - makefile.vs (for nmake)
• Windows: Visual Studio project file - vs/vs2017
• Linux amd64: GNU GCC (>=4.6)
• Linux amd64: Clang (>=3.2)
• Linux arm64: 64 bits aarch64 ARMv8: gcc (>=6.3)
• Linux arm64: 64 bits aarch64 ARMv8: clang
• MaxOS: XCode (>=9)
• PowerPC ppc64le (incl. SIMD): gcc (>=8.0)

Run Length Encoding References:

• Real-Time Compression of IEC 61869-9 Sampled Value Data
• Understanding Compression: Data Compression for Modern Developers
• Understanding Compression inl. Run Length Encoding
• Entropy Coder Benchmark

Last update: 1 APR 2023