Youzan Nsq Versions Save

• fix a possible panic for delayed queue
• adjust req strategy on the delayed queue
• optimize the stats API
• fix unexpected reset reader while reading disk
• retry delivering msg to the client, regardless of the reset result
• fix channel creation returns 200 when the topic is not found

1. fix load channel should not persist the init meta which will cause the state lost for the channel if exit by unexpected.
2. handle the message which timeout too much to avoid blocking too long
3. add the experimental kv-index for the topic (disabled by default).
4. clean check max available if can not greedy clean directly
5. return an error if update channel state or delete channel failed to sync

1. fix: etcd watched max index which may cause lost newest event.
2. fix: should continue to pull data from the leader if EOF on leader read while not disable write
3. optimize the skipped channel to avoid read disk data
4. fix: select node by the least load factor. Thanks to @Damon07
5. fix: handle the possible conflict in the inflight between normal and delayed message

1. add API to get a message from the delayed queue and the data tool also support search by the msg-id in the delayed queue.
2. optimize the delayed queue empty
3. optimize the topic channel meta save and history pub stats
4. some optimize for a large number of topics in a single cluster

1. add limiter for pub and fail fast if too much waiting
2. add rate limiter for consuming the large messages
3. allow setting the max ready for client
4. wakeup quick if some re-queued messages waiting in the map
5. optimize and refactor JSON parser
6. avoid read backend while too much deferred in mem to reduce memory usage

1. allow cached the etcd request from the lookup
2. add some options to control the memory usage
3. add Prometheus metric for nsqd
4. timer and ticker leak fix and optimize to improve performance and CPU usage
5. support disable channel auto-create
6. optimize the mpub to allow group commit

1. optimize the delayed queue sync between ISR
2. add the restriction on the max delayed queue count
3. use boltdb for disk write queue meta
4. add config to always write queue meta to old the file to allow rollback to the old version
5. use NoFreelistSync to optimize the delayed queue write performance
6. allow consume committed data on the disabled leader which can speed up the consumer recovery if half of ISR nodes lost.
7. fix pub timeout timer clean

1. add the restriction on the max delayed queue count 2.fix draining channel should check the delayed-type while clear the counter for delayed 3.use Array Type for the boltdb which seems better for delete operation

fix the delayed message may fail to continue consume if the channel does empty or skip.

1. log the trace sub cost time. Thanks to @zhanghuanzhong
2. force fix reader to queue start if read cursor is invalid while update end
3. optimize the full sync for delayed queue
4. optimize consumer latency in some special cases
5. clean invalid commit log data if init failed with old data
6. force fix data if start with non-leader or single replica