Sonic Versions Save

🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.

5 years ago

RocksDB has been bumped to v5.18.3, which fixes a dead-lock occuring in RocksDB at scale when a compaction task is ran under heavy disk writes (ie. disk flushes). This dead-lock was causing Sonic to stop responding to any command issued for the frozen collection. This dead-lock was due to a bug in RocksDB internals (not originating from Sonic itself) [@baptistejamin, 19c4a104a6d6aaed1dd9beb2e51d2639627825cd].
Reworked the FLUSHB command internals, which now use the atomic delete_range() operation provided by RocksDB v5.18 [@valeriansaliou, 660f8b714d968400fb9f88a245752dca02249bf7].
Added the LANG(<locale>) command modifier for QUERY and PUSH, that lets a Sonic Channel client force a text locale (instead of letting the lexer system guess the text language) [@valeriansaliou, #75].
The FST word lookup system, used by the SUGGEST command, now support all scripts via a restricted Unicode range forward scan [@valeriansaliou, #64].

5 years ago

A store acquire lock has been added to prevent 2 concurrent threads from opening the same collection at the same time [@valeriansaliou, 2628077ebe7e24155975962471e7653745a0add7].

5 years ago

A superfluous mutex was removed from KV and FST store managers, in an attempt to solve a rare dead-lock occurring on high-traffic Sonic setups in the KV store [@valeriansaliou, 60566d2f087fd6725dba4a60c3c5a3fef7e8399b].

5 years ago

Reverted changes made in v1.1.5 regarding the open files rlimit, as this can be set from outside Sonic [@valeriansaliou, f6400c61a9a956130ae0bdaa9a164f4955cd2a18].
Added Chinese Traditional stopwords [@dsewnr, #87].
Improved the way database locking is handled when calling a pool janitor; this prevents potential dead-locks under high load [@valeriansaliou, fa783728fd27a116b8dcf9a7180740d204b69aa4].

5 years ago

Added the server.limit_open_files configuration variable to allow configuring rlimit [@valeriansaliou].

5 years ago

Automatically adjust rlimit for the process to the hard limit allowed by the system (allows opening more FSTs in parallel) [@valeriansaliou].
Added Kannada stopwords [@dileepbapat].
The Docker image is now much lighter [@codeflows].

5 years ago

Rework Sonic Channel buffer management using a VecDeque (Sonic should now work better in harsh network environments) [@valeriansaliou, 1c2b9c8fcd28b033a7cb80d678c388ce78ab989d].
Limit the size of words that can hit against the FST graph, as the FST gets slower for long words [@valeriansaliou, #81].

5 years ago

FST graph consolidation locking strategy has been improved even further, based on issues with the previous rework we have noticed at scale in production (now, consolidation locking is done at a lower-priority relative to actual queries and pushes to the index) [@valeriansaliou, #68].

5 years ago

FST graph consolidation locking strategy has been reworked as to allow queries to be executed lock-free when the FST consolidate task takes a lot of time (previously, queries were being deferred due to an ongoing FST consolidate task) [@valeriansaliou, #68].
Removed special license clause introduced in v1.0.2, Sonic is full MPL 2.0 now. [@valeriansaliou]

5 years ago

Breaking: Change how buckets are stored in a KV-based collection (nest them in the same RocksDB database; this is much more efficient on setups with a large number of buckets - v1.1.0 is incompatible with the v1.0.0 KV database format) [@valeriansaliou].
Bump jemallocator to version 0.3 [@valeriansaliou].