A very fast scalar implementation for Frame Of Reference integer compression.
An ANSI C library with a fast scalar (non-SIMD) implementation for Frame of Reference (FOR) integer compression. It works on Linux, Microsoft Windows and most likely all other sane systems.
FOR can compress sorted and unsorted integer sequences.
In addition, the library can perform operations directly on compressed data:
#define LEN 100
uint32_t in[LEN] = {0};
uint8_t out[512];
// Fill |in| with numbers of your choice
for (int i = 0; i < LEN; i++)
in[i] = i;
// Now compress; can also use for_compress_sorted() if the numbers
// are sorted. This is slightly faster.
uint32_t size = for_compress_unsorted(&in[0], &out[0], LEN);
printf("compressing %u integers (%u bytes) into %u bytes\n",
LEN, LEN * 4, size);
// Decompress again
uint32_t decompressed[LEN];
for_uncompress(&out[0], &decompressed[0], LEN);
It can't be more simple:
make
To run the tests:
./test
I use this library to compress 32bit integers for upscaledb, a very fast embedded key/value store (see http://upscaledb.com).
If you would like me to add your application to this list then please send me a mail at [email protected].
Apache License, Version 2.0
This library only works with little-endian CPUs.
Tested on Linux and Windows (Visual Studio 2013). Porting it should not be difficult.
This work is based on Daniel Lemire (http://lemire.me)'s ideas and implementation at https://github.com/lemire/FrameOfReference.
For further information, see