Kvrocks Versions Save

Here is a comprehensive list of changes in this release compared to 2.0.4

New features

• Implement zrevrangebylex command (#381)

• Use some rocksdb new feature (#395)

  • Support blob db (key-value separation) which requires RocksDB version greater than 6.18
  • Enable partitioned index/filter for SST and hash index for data block
  • Enable whole key filter for memtable
  • Remove dynamic resize block and SST
  • Only apply compression to SST which level is more than 2

  Please note that, if your rocksdb version is lower than 6.6.0, it can't reload the data of new version, you should update cautiously.

• Implement CAS and CAD commands for string type (#415)

• Support to use row cache of RocksDB (#407)

Improvements

• Add rocksdb ops statistics into the info command (#387) These statistics can more truly reflect the load of rocksdb.
• Add the maxclients to the info command output (#391)
• Add write stall/stop stats to info command output (#396)
• Make kvrocks2redis work (#404)
• Adjust the max open file limit according to the max clients and rocksdb max open files (#418)

Related projects

• Kvrocks exporter, a fork of oliver006/redis_exporter to export the kvrocks metrics, https://github.com/KvrocksLabs/kvrocks_exporter

Here is a comprehensive list of changes in this release compared to 2.0.3

New features

• Support LUA script commands (#363) (#369) Currently, support EVAL, EVALSHA, and SCRIPT commands, but SCRIPT KILL and DEBUG subcommands are not supported.

Improvements

• Allow metadata and subkey CF to share a single block cache (#365) (#373)
• Add master_repl_offset in INFO command output (#374)

Bugfixes

• Fixed dynamic resize target_file_size_base bug (#368) Before we only apply this strategy to the default CF(i.e. subkey CF)

If your kvrocks instances are 2.0.2 and use redis-sentinel to implement HA, you should upgrade urgently, please see #352 for details.

Here is a comprehensive list of changes in this release compared to 2.0.2

Improvements

• Use the rocksdb MultiGet interface in the MGET command (#331)
• Avoid useless space allocation in SetBit (#338)
• Auto set master/replica replication by cluster topology (#356)
• Support client kill type subcommands to adapt redis-sentinel (#352) (#361) From 2.0.2, we support MULTI-EXEC, but don't support client kill type $type, so MULTI-EXEC will be rollbacked when failing to parse unknown kill type in client command, and cause redis-sentinel fails to failover.

Bugfixes

• Fix SETNX and MSETNX commands can't guarantee atomicity (#337)
• Fix the replicas won't reconnect when failing to connect with master (#344)
• Fix master pauses synchronizing with replicas (#347)
• Fix wrong mater port if enabling master-use-repl-port (#360)

We design new cluster solution for kvrocks, but it is experimental, don't recommend to deploy in production environments. And because in cluster mode, to be efficient to migrate keys based on slot when scale, we encode key with slot id, but in order to be compatible with the old version, we don't do that in standalone mode, so data is not compatible between standalone mode with cluster mode, you must migrate data if you want to change mode, otherwise, kvrocks will make data corrupt. In order to better support the cluster mode, we are developing a meta server to manage kvrocks cluster, coming soon!

Here is a comprehensive list of changes in this release compared to 2.0.1

New features

• Support rename command in config file (#272)
• Support transaction (#285) Support MULTI, EXEC, DISCARD command, but don't support WATCH and UNWATCH commands.
• Support the auto-resize-block-and-sst config directive (#289)
• Support to resize rocksdb.block_size automatically (#294)
• Support cluster mode (#302) Similar with redis cluster mode, but use CLUSTERX command to set cluster topology (#324).

Data encoding changes

• Encode key with slot id (#291) (#330) In cluster mode, to be efficient to migrate keys by slot when scale, we encode key with slot id, but in standalone mode, we don't. That is to say, data is not compatible between standalone mode with cluster mode, you must migrate data if you want to change mode, otherwise, kvrocks will make data corrupt.

Bug fixes

• Set TCP_KEEPALIVE for replication connection (#284) Replicas can handle network failure with master, before this commit, network failure may result in unrecoverable replication interrupt.
• Fix ZSET can't add the same member with different scores (#298)
• Make CONFIG SET command thread safe (#310)
• Fix LTRIM only delete first item, LREM can't correctly remove repeated items (#314)

Improvements

• HSET command supports to set multiple fields (#274)
• Allow to use host in the master-slave replication (#301)
• Adapt redis sentinel ping check (#316)

Here is a comprehensive list of changes in this release compared to 2.0.0

New features

• Support COMMAND command (#251)
• Support to purge backups on fullsync to reduce disk space usage (#245)

Bug fixes

• Fix uninitialized Rocksdb.level0_stop_writes_trigger (#236)
• Fix condition race for ReclaimOldDBPtr (#246)
• Fix data race for accessing database in some commands (#253) (#261) In spop, zremrangebylex, zremrangebyscore commands, the lock guard for accessing the database may not work actually.
• Fix returning wrong string range in GETRANGE command (#254)
• Fix skipping wrong CURRENT file when full synchronization (#260) Resuming broken transfer based files may be not available if replicas skip transferred wrong CURRENT file of RocksDB.

Improvements

• Replicas can execute publish command (#238)
• Optimizations for avoiding unnecessary data copy (#239) (#247)
• Allow to run the kvrocks without assigning the config file (#264)

Dependencies

• Upgrade the jemalloc version to the special commit (#265) To be able to compile on macOS, upgrade jemalloc to the production test commit which mentioned in jemalloc/jemalloc#2060.

Here is a comprehensive list of changes in this release compared to 1.3.2

Bug fixes

• Fix rocksdb can't auto resume after no space error (#229)
• Fix uninitialized Rocksdb.level0_stop_writes_trigger (#236)
• Fix condition race for ReclaimOldDBPtr (#246)
• Fix returning wrong string range in GETRANGE command (#254)
• Fix data race for accessing database in some commands (#253) In spop, zremrangebylex, zremrangebyscore commands, the lock guard for accessing the database may not work actually.

Improvements

• Replicas can execute publish command (#238)

Kvrocks 2.0 brings some new useful features that solve actual problems in the production environment. Please note that this version instance still can replicate from old version master but old version replicas can't replicate new version master. Otherwise, we start to use new logo, thanks @tianlingyu1997

Here is a comprehensive list of changes in this release compared to 1.3.2

New features

• Don't use extra port for replication (#200)
• Use checkpoint to implement data snapshot when full replication (#208) Reduce disk space and bandwidth usage and decrease the time of replicas recovery during full synchronization.
• Use checkpoint to implement kvrocks backup (#232)
• Replicas can empty db before fullsync (#233) Replicas use less disks space if enabled, but have a risk to lose data.

Command changes

• Add QPS and input/output kbps for INFO command (#213)
• Add pubsub_patterns for INFO command (#234)
• Slowlog omits args or argument string if too many or big (#215)

Cluster

• Remove codis support (#217) From now, we doesn't support codis, you can adopt pre-sharding to implement cluster, and we are developing new cluster solution that is similar with redis cluster mode, please see #219.

Bugfixes

• Fix kvrocks can't auto resume after no space error (#229)

Dependencies

• Upgrade rocksdb to latest tag v6.19.3 (#226)

Here is a comprehensive list of changes in this release compared to 1.3.1

Bug fixes

• Fix incorrect used_db_size in INFO command (#204)
• Fix the SST file creation time maybe 0 in some unknown conditions (#211)
• Fix get corruption writebatch data after psync restart (#221)

Improvements

• Optimize TCP keepalive detection time (#201)

Here is a comprehensive list of changes in this release compared to 1.2.0

License

• Change the license from MIT to BSD-3-Clause

New features

• Support to set separate log directory to store server logs (#155)
• Implement PSETEX command (#156)
• Implement MSETNX command (#165)

Improvements

• Improve the precision of the INCRBYFLOAT/HINCRBYFLOAT command to double (#169)
• Support +INF/-INF for the score of zset type members (#172)
• The UNSUBSCRIBE and PUNSUBSCRIBE commands support multiple channels or patterns (#174)

Behavior changes

• The UNSUBSCRIBE and PUNSUBSCRIBE commands have the same reply as Redis (#174)
• The same message format as Redis for pattern subscribers when publishing message (#175) Message format is from 'message channel message' to 'pmessage pattern channel message'

Bug fixes

• Fix skipping some SST files when picking compaction files (#154)
• Handle negative expire time in set command to avoid deleting keys (#156)
• List type command doesn't rewrite other types of data (#157)
• Only set positive expire time when migrating key (#160)
• Don't store an empty key for setrange command (#161)
• Fix SETBIT/GETBIT bit offset out of range (#164)
• Fix INCRBY/INCRBYFLOAT command operates on wrong value and precision check (#167)
• Fix HINCRBY/HINCRBYFLOAT command operates on wrong value and precision check (#168)
• RPOPLPUSH command doesn't operate non list type (#170)
• LINSERT command returns -1 if don't find index (#170)
• Fix LTRIM command range calculation when start is a very large negative number (#170)
• SMOVE command doesn't operate non set type key (#171)
• Fix ZREMRANGEBYRANK command corrupts the size of zset (#172)
• SUBSCRIBE and PSUBSCRIBE commands return the sum of subscription and psubscription (#174)
• Fix PUNSUBSCRIBE command doesn't take effort (#174)
• Allow setting slowlog-log-slower-than to -1 to disable the slowlog (#178)