M3 Versions Save

Changelog

Features and Performance

• M3DB: Optimize snapshotting by 30x or more which can speedup bootstrapping for workloads with large snapshots. (#4093)

Bug Fixes

• M3DB: Fix multi-segment field iterator support of double underscore prefixed fields alphanumerically before __m3ninx_id. (#4095)
• M3DB: Proactively GC Index entries belonging to a closed shard after cluster topology change. (#4094)

Changelog

Bug Fixes

• M3Coordinator: Guard against duplicate rollup tags in aggregation matching logic. (#3950)

Performance

• M3DB: Improve index query performance by avoid cloning of roaring bitmap in conjunctionSearcher. (#3948)
• M3DB: Improve bootstrapping memory profile by fixing entry leak in series ref resolver. (#3980)
• M3DB: Provide better balanced shard placement by fixing unbalanced initial shard allocation. (#4020)

Changelog

Bug Fixes

• M3Coordinator: Do not Close singleton MessageProcessors when closing connections. This fixes a panic introduced that affects M3Coordinator -> M3Aggregator communication. (#3934)

Changelog

Features

• M3Query: Add write endpoint support for M3-Map-Tags-JSON header in InfluxDB path (#3816)
• M3Query: Add support for last_over_time in M3Query engine (#3884)
• M3Aggregator: Add p75/p25 as aggregation options (#3867)

Bug Fixes

• M3DB: Fix M3TSZ to be deterministic when encoding high precision values (3872)
• M3DB: Gracefully handle reads including documents with stale index state (3905)

Performance

• M3Aggregator: Rework close and remove persitFlushTimesEvery semantics in leader flushing in favour of always persisting shard flush times on a successful flush for optimized graceful failovers (#3890)
• M3DB: Optimize filesetFiles function during bootstrapping for namespaces with long retentions to prevent CPU spikes (#3900)
• M3DB: Avoid loading blocks in memory for namespaces with snapshots disabled during bootstrapping to reduce memory usage (#3919)

Changelog

Features

• M3Coordinator: Add support for Prometheus Remote Write storage backend for sending aggregated and unaggregated metrics (#3742, #3768, #3783, #3791, #3814, #3777)
• M3Coordinator: Add support for InfluxDB write endpoint GZip compression, setting timestamp precision and allowing an empty request body (#3373)
• M3DB: Add SYSCTL_VM_MAX_MAP_COUNT env var for sysctl-setter sidecar allowing for custom VM max map count (#3689)

Bug Fixes

• M3DB: Fix writes briefly degrading when creating a new namespace due to coarse lock acquisition (#3765)
• M3DB: Fix compiled regexp DFA cache eviction on full bug that can lead to slow memory leak with large number of unique regexps (#3806)

Performance

• M3Coordinator: Update default M3Msg retry initial backoff from 1s to 5s to reduces timeout and retries in large clusters (#3820)
• M3DB: Fix performance of reverse index queries that cover huge time ranges (#3813)

Changelog

Features

• M3Query: Support Prometheus matchers with match[] URL parameters in label endpoints (#3180)
• M3Query: Support Prometheus start and end time URL parameters for label and series metadata endpoints (#3214)
• M3Query: Add Graphite functions and update functions with new arguments that were missing (#3048, #3367, #3370, #3145, #3149, #3142, #3469, #3484, #3545, #3576, #3582, #3583, #3521, #3602, #3641, #3644, #3648)
• M3Query: Fix Graphite treatment of ** to allow to match an empty segment instead of one or more (#3366, #3593)
• M3Query: Add M3-Limit-Max-Range header to optionally truncate time range of queries (#3538)
• M3Coordinator: Add ability to use an exclude by rollup rule to rollup metrics without specific dimensions (#3318)
• M3DB: Use better heuristics to cap the series and aggregate query limits that individual DB nodes apply for a query so in larger clusters the query can be clamped earlier (#3516, #3518, #3519, #3520, #3527)
• M3DB: Add repair option full_sweep and ability to force a repair via API call (#3573, #3550)

Bug Fixes

• M3DB: Fix aggregate series metadata query limits (#3112)
• M3Coordinator: Make bad aggregated namespace headers return bad request status code instead of internal server error (#3070)
• M3Coordinator: Propagate Require-Exhaustive parameter for aggregate series metadata queries (#3115)
• M3Query: Add determinism to Graphite sort and reduce functions (#3164)

Performance

• M3DB: Rearchitect index segments to compact and expire series on block rotation instead of build a new segment for new block (#3464)
• M3DB: Add postings list cache for searches and repopulate during active block index segment compaction before segment made visible for queries (#3671)
• M3DB: Avoid allocating index entry fields per series and read from backing mmap directly (#3050, #3062, #3057)
• M3DB: Avoid allocating series IDs when read from disk (#3093)
• M3DB: Improve speed of tag byte reuse from ID for tags by speeding up search (#3075)
• M3DB: Improve speed of bootstrap by using StreamingReadMetadata API for reads from disk (#2938)
• M3DB: Improve speed of bootstrap by using an asynchronously evaluated series resolver API that can be written to while bootstrapping reliably (#3316)
• M3DB: Add limits for total series being read at any one time globally (#3141)
• M3DB: Restrict the time a query can hold an index worker to help allow small queries to continue to execute while larger ones are paused and resumed (#3269)
• M3DB: Use adaptive WriteBatch allocations to dynamically match workload throughput and batch sizes (#3429)
• M3Coordinator: Improve rule matching speed by improving per element rule matching and disabling cache which puts locks in the hot path (#3080, #3083)
• M3Query: Improve speed of M3TSZ decoding by using 64 bit operations (#2827)
• M3Query: Improve speed of M3TSZ decoding by using int64 type xtime.UnixNano instead of time.Time (#3515)
• M3Query: Improve speed of quorum reads by improving multi-replica iterator (#3512)

Changelog

Features

• M3Coordinator: Add /ready endpoint for readiness probe which checks current write/read consistency level achievability (#2976)
• M3Coordinator: Add per endpoint status code response codes and latency metrics (#2880)
• M3Coordinator: Add Graphite carbon ingest latency metrics (#3045)
• M3Coordinator: Add Graphite carbon ingest rule matcher contains to compliment regexp for faster matching (#3046)
• M3Coordinator: Return 504 errors on timeout to downstream M3DB nodes or other cross-region coordinators (#2886)
• M3Coordinator: Validate placements when using the raw placement upsert endpoint unless force set is specified (#2922)
• M3Query: Add Graphite powSeries function (#3038)
• M3Query: Add Graphite support for ** with metric path selectors (#3020)
• M3DB: Add ability to configure regexp DFA and FSA limits (#2926)
• M3DB: Add Alibaba Cloud storage class Kubernetes manifest for disk provisioning in Aliyun (#2908)

Bug Fixes

• M3Coordinator: Always set content type JSON for error responses (#2917)
• M3Query: Fix invalid query resulting in 500 instead of 400 (#2910)
• M3Query: Allow Graphite variadic functions to omit variadic args (#2882)

Performance

• M3DB: Skip out of retention index segments during bootstrap (#2992)

Documentation

• All: Add clustering getting started guides for both Kubernetes operator and binaries deployment (#2795)

Changelog

• M3DB: Add ability to force enable cold writes from config (#3002)

Changelog

• M3Coordinator: Add readiness probe for probing current consistency level achievability (#2976)
• M3DB: Fix client config respecting value for connect consistency (#2914)

Changelog

3746bf0e3 [dbnode] Add validation to ensure index claims is singleton (#2877) a0a96a77b [dbnode] Default HostID and Discovery config for minimal config (#2876) 251dc3d29 [dbnode] Add claims for index segments volume index (#2846) 5b5c05022 [dbnode] Remove namespaces from example config and integration tests (#2866) a0a4c614b [dbnode] Resurrect flaky test skip (#2868) 47cde347f [aggregator] Fix checkCampaignStateLoop (#2867) 2d8dfe563 [dbnode] implement deletion method in namespace kvadmin service (#2861)