Kafkajs Versions Save

[1.15.0] - 2020-11-24

Added

• Initial work for static membership #888
• Add consumer instrumentation event: received unsubscribed topics #897
• Add option for admin.fetchOffsets to resolve the offsets #895
• Add ACL functions to admin client #697
• Add admin.deleteTopicRecords #905
• Emit GROUP_JOIN event on stale partition assignments #937

Changed

• Added properties to error classes typescript types #900
• Make header value type definition possibly undefined #927
• Bump API versions for client-side throttling #933
• Add UNKNOWN_TOPIC_OR_PARTITION check for addMultipleTargetTopics #938

Fixed

• Fix describe/alter broker configs (introduced ConfigResourceTypes) #898
• Fix record batch compression masking (fix ZSTD compression) #912
• Prevent inflight's correlation id collisions #926
• Fix ACL, ISocketFactory and SaslOptions type definitions #941 #959 #966
• Fix deadlock on the connection onError handler #944
• Fix deadlock on the connection onTimeout handler #956
• Remove nested retriers from producer #962 (fixes #958 #950)

[1.14.0] - 2020-09-21

Added

• Support Produce v6 protocol #869
• Support Produce v7 protocol (support for ZSTD compression) #869
• Broker rediscovery with config.brokers parameter taking a callback function #854 #882

Changed

• Remove long.js in favor of BigInt #663
• Remove allowExperimentalV011 flag #847

Fixed

• Only commit offsets on eachMessage failures if autoCommit is enabled #866
• Fix consumer offsets not committed if consumer stop was invoked right after the batch process #874
• Remove brokers with closed connections from the brokers list #878
• Type improvements and fixes #877

[1.13.0] - 2020-09-10

This is a large release containing fixes and features from a multitude of contributors. Some notable changes:

• Dropped support for Node 8. Given that Node 8 was EOL in the beginning of 2020, for the overwhelming majority of people this should not be an issue.
• For Typescript users this release may require you to make minor changes, due to #764, which changes the type of KafkaMessage.value from Buffer to Buffer | null. This was in fact always the behavior at runtime, so this change is just fixing a bug in the types - but it will require you to handle the value being null for your project to build.
• Thanks to @ankon, we now support all the necessary functionality to enable Rack-Aware Follower Fetching to cut data transfer costs and reduce latency.
• Several changes to improve performance! Some larger initiatives, such as eagerly processing fetched batches in parallel, and some more contained optimizations such as reducing the amount of buffer copying during socket data buffering. We hope that these changes will have a noticeable effect on your applications, and look forward to hearing your feedback.

Added

• Add listGroup method to admin interface #645
• Add describeCluster method to admin client #648
• Add createPartitions method to admin client #661
• Add deleteGroups method to admin client #646
• Add listTopics method to admin client #718
• Add describeGroups method to admin client #742
• Allow to handle consumer retry failure at the user level #643
• Support Fetch v8 protocol (including client-side throttling) #776
• Support Fetch v9 protocol #778
• Support Fetch v10 protocol #792
• Support Fetch v11 protocol #810
• Support JoinGroup v3 and v4 protocol #801
• Oauthbearer support #680
• Add new protocol errors #824
• Add versioning to docs #835
• Add fetch topic offsets by timestamp #604
• Support LOG_APPEND_TIME record timestamps #838
• Suppress JoinGroup V4+ response error log when memberId is empty #860

Changed

• Replace fetch promise all with async generator #570
• Improve balance in the RoundRobinAssigner #635
• Add single requestTimeout runner instead of setTimeout per request #650
• Provide the subscribed topics to the protocol() function #545
• Only wait for the lock when there are enqueued batches #670
• Update consumer default retries to 5 #720 (related to #719)
• Simplify and speed up SeekOffsets #668
• Remove maxInFlight option from default retry and moved into Producer ad Consumer #754
• Use addMultipleTargetTopics instead of looping over multiple calls to addTargetTopic #748
• Only disconnect the consumer and producers if they were created #784
• Resolve socket requests without response immediately when they have been queued #785
• Move default request timeout from connection to cluster #739
• Simplify the BufferedAsyncIterator #671
• Ensure fair fetch response allocation across topic-partitions #859

Fixed

• Type improvements and fixes #636 #664 #675 #722 #729 #758 #799 #813 #757 #749 #764 #828 #843 #839
• Remove invalid topics from targetTopics on INVALID_TOPIC_EXCEPTION #666
• Fix network buffering performance problems #669
• Delete the entry for the waiter when the timeout is reached #694
• Fix encoder instanceof issue with Encoder #685
• Fix default retry for consumer #719
• Runner#waitForConsumer uses consuming stop event instead of timers #724
• Fix unhandled rejections #714 #797
• Make Array shuffle test pass in Node >= 11.0.0 #740
• Use setImmediate when scheduling calls of scheduleFetch #752
• Improve offset commit handling #775
• Use the length from the message to pre-allocate the result array #771
• Fix application lock in case of errors before the connection is established #780
• Add isNaN check for concurrency limit #787
• Fix admin client createTopics timeout #800
• Avoid repeated costly copies of buffers when working with encoders #811
• Fix fall-back retry config for producer #851

[1.12.0] - 2020-01-30

Added

• Force refresh of metadata when topic is not present #556
• Expose ConsumerRunConfig type #615
• Randomize order of seed brokers #632

Changed

• Support TLS SNI by default #512
• Changed typing of logLevel argument of logCreator #538
• Add type boolean in ssl KafkaConfig #557
• Allow logging Fetch response payload buffers #573
• Remove default null for logCreator #595
• Add error names to ensure error names work with webpack + uglify #602
• Merge TopicMessages by topic in producer sendBatch #626

Fixed

• Skip control records without auto-resolve #511
• Handle empty member assignment #567
• Only fetch for partitions with initialized offsets #582
• Get correct next offset for compacted topics #577
• TS type definition for removing instrumentation event listeners #608
• Fixed IHeaders definition to accept plain strings #547
• Make TS type ProducerBatch fields optional #610
• Fix typings for logger getters #620

[1.11.0] - 2019-09-30

Added

• Add Typescript SASLMechanism type to definitions #477
• Allow SASL Connections to Periodically Re-Authenticate #496

Changed

• Throw validation error when the broker list is empty #460
• Improve the encoder to avoid copying unnecessary bytes #471
• Throw an error on subscription changes for running consumers #470
• Default throttle_time_ms to 0 if missing in ApiVersions response #495
• Remove normalisation of the password when using SCRAM mechanism #505

Fixed

• Fix built-in partitioners type definition error #455
• Detect replaced brokers on refresh metadata #457
• Make NodeJS REPL get correct randomBytes() #462
• Fix IHeaders type definition (from string to Buffer) #467
• Rename Typescript definitions from ResourceType to ResourceTypes #468
• Update Typescript definitions to make configNames optional in ResourceTypes #474
• Fix transactionState.ABORTING value #478

[1.10.0] - 2019-07-31

Added

• Allow the consumer to pause and resume individual partitions #417
• Add consumer.commitOffsets for manual committing #436
• Expose consumer paused partitions #444

Changed

• Removed unnecessary async code #434
• Use SubscriptionState to track member assignment #429

Fixed

• Improve type compatibility with @types/kafkajs #416
• Fix fetchTopicMetadata return type #433

[1.9.3] - 2019-06-27

Fixed

• Fix AWS-IAM mechanism name #411 #412
• Fix TypeScript types for topic subscription with RegExp #413

[1.9.2] - 2019-06-26

Fixed

• Fix typescript types for Logger, consumer pause and resume, eachMessage and EachBatch interfaces #409

[1.9.1] - 2019-06-25

Fixed

• Fix typescript types for SSL, SASL and batch #407

[1.8.1] - 2019-06-25

Fixed

• Make sure runner has connected brokers and fresh metadata before it starts #404