Alpakka Kafka 3.0 features the upgrade to the current Kafka client library 3.0.0 which is backwards compatible with Kafka brokers of older versions. Alpakka Kafka 3.0 is published for Scala 2.13.

Alpakka Kafka does not change its APIs, even previously deprecated APIs are kept. The major version upgrade is motivated by the Kafka clients upgrade and some changes to its dependencies.

Changes

• Kafka client 3.0.0 does not require Jackson anymore
• With being Scala 2.13 only the Scala Collection Compatibility library was removed

Upgrading to Alpakka Kafka 3.0

We recommend you ensure to be on current versions of the Alpakka Kafka dependencies before upgrading.

• Scala 2.13.8
• Akka 2.6.18

Full Changelog: https://github.com/akka/alpakka-kafka/compare/v2.1.1...v3.0.0-RC1

Changes

Alpakka Kafka core

• Fix context propagation in CommitCollectorStage #1395 by @ygree

Tests

• Fix context propagation in CommitCollectorStage #1395 by @ygree

Documentation

• Release Notes 2.1.1 #1397 by @ygree

Changes

• sbt 1.5.1 (was 1.4.9) #1366 by @scala-steward
• sbt-ci-release 1.5.7 (was 1.5.6) #1359 by @scala-steward
• metrics-core 4.1.21 (was 4.1.18) #1363 by @scala-steward
• mockito-core 3.9.0 (was 3.8.0) #1365 by @scala-steward
• testcontainers-kafka 1.15.3 (was 1.15.2) #1367 by @scala-steward

Alpakka Kafka core

• 2.1.0 release notes #1369 by @seglo
• Akka typed first documentation user experience #1370 by @seglo

Tests

• Akka typed first documentation user experience #1370 by @seglo

Documentation

• 2.1.0 release notes #1369 by @seglo
• Akka typed first documentation user experience #1370 by @seglo

Changes

• testcontainers kafka 1.15.2 (was 1.15.1) #1345 by @scala-steward
• sbt 1.4.9 (was 1.4.5) #1352 by @scala-steward
• scala-collection-compat 2.4.3 (was 2.4.2) #1351 by @scala-steward
• scala-logging 3.9.3 (was 3.9.2) #1350 by @scala-steward
• fix typo in GH action workflow #1346 by @octonato
• scala-library 2.12.13 (was 2.12.11) #1342 by @scala-steward
• scala-collection-compat 2.4.2 (was 2.2.0) #1344 by @scala-steward
• mockito-core 3.8.0 (was 3.7.7) #1341 by @scala-steward
• junit-jupiter-api 5.7.1 (was 5.7.0) #1340 by @scala-steward
• jupiter-interface, ... 0.9.1 (was 0.8.4) #1339 by @scala-steward
• junit 4.13.2 (was 4.13.1) #1338 by @scala-steward
• metrics-core 4.1.18 (was 4.1.17) #1337 by @scala-steward
• paradox-theme-akka, ... 0.38 (was 0.37) #1335 by @scala-steward
• Update scaladex badge #1328 by @seglo
• scalapb-runtime 0.10.11 (was 0.10.9) #1320 by @scala-steward
• metrics-core 4.1.17 (was 4.1.16) #1321 by @scala-steward
• mockito-core 3.7.7 (was 3.7.0) #1324 by @scala-steward
• jupiter-interface, ... 0.8.4 (was 0.8.3) #1322 by @scala-steward

Alpakka Kafka core

• Upgrade Akka 2.6.14 and Kafka 2.7.0 #1355 by @seglo
• Add configurable protection against server-bug induced resets #1299 by @jyates

Tests

• Upgrade Akka 2.6.14 and Kafka 2.7.0 #1355 by @seglo
• Increase consumer group summary timeout in build #1329 by @seglo
• Add configurable protection against server-bug induced resets #1299 by @jyates
• Migrate build to GitHub actions #1307 by @seglo

Alpakka Kafka Testkit

• Upgrade Akka 2.6.14 and Kafka 2.7.0 #1355 by @seglo
• Don't depend on scalatest in testcontainer singleton #1330 by @seglo
• Add ProducerResultFactory.multiResult helper for MultiMessage #1319 by @ashendon

Documentation

• Release notes 2.1.0-RC1 #1357 by @seglo
• Upgrade Akka 2.6.14 and Kafka 2.7.0 #1355 by @seglo
• Migrate to sonatype snapshot/release repositories #1353 by @seglo
• Add configurable protection against server-bug induced resets #1299 by @jyates
• Release notes v2.0.7 (master) #1317 by @seglo
• Migrate build to GitHub actions #1307 by @seglo

Alpakka Kafka Testkit

• Increase default cluster start timeouts and make configurable (backport) #1311 by @seglo
• Use restart script #1310 by @seglo

Release notes in the Alpakka Kafka documentation

Notable changes since 2.0.5

The most relevant change in this release is the update of TestContainers to 1.15.1 (see below)

• Handle any combination of Committables in CommitObservationLogic #1262 by @seglo
• Fix message-batch loss when rebalancing partitioned sources #1263 by @jhooda
• Producer.committableSink: offsets not committed, when MultiMessage is empty #1200 by @herzrasen

Alpakka Kafka TestKit

• Update default CP version and make docker image/tag configurable #1287 by @seglo
• Testkit: Make Testcontainer logs visible #1281 by @seglo
• Testcontainers Kafka 1.15.1 (was 1.14.3) #1257 by @scala-steward
• TestKit: make consumerDefaults parameterless again #1292

Dependencies

• jackson-databind 2.10.5.1 (was 2.10.5) #1270 by @scala-steward
• Add note about provided jackson dependency #1245 by @seglo

Release notes in the Alpakka Kafka documentation

The Alpakka Kafka 2.1 series features

• Kafka 2.6.0 client
• Akka 2.6.x (dropped Akka 2.5)
• Scala 2.12 and 2.13 (dropped Scala 2.11)
• Testkit: use ScalaTest 3.1.x
• Testkit: default to Confluent Platform 6.0.0
• Testkit: no longer support Embedded Kafka (Kafka 2.6.0 can't be safely embedded in Scala applications)

Release notes in the Alpakka Kafka documentation

Changes

• Generate version information when publishing artifacts #1188 by @marcospereira

Alpakka Kafka core

• RestrictedConsumer: correct deprecation since version #1197 by @ennru
• Allow typed Actor Systems in SendProducer and DiscoverySupport #1192 by @ennru
• Deprecate RestrictedConsumer.committed(TopicPartition) #1178 by @seglo
• Cleanup build warnings #1177 by @seglo
• Order logging params for RetriableException consumer failures #1168 by @seglo

Tests

• Reduce bloat of log when testsuite log reported in failed travis build #1182 by @seglo
• Simplify consuming stream for AssignmentSpec "consume from the specified topic pattern" test #1181 by @seglo
• Cleanup testcontainers docs snippets #1180 by @seglo
• Cleanup build warnings #1177 by @seglo

Alpakka Kafka Testkit

• Cleanup build warnings #1177 by @seglo

Documentation

• 2.0.5 Release notes #1199 by @ennru
• Run & materialize stream in Akka Cluster Sharding Example #1190 by @seglo
• docs: update Akka version #1198 by @ennru

Updates

• scalapb-runtime 0.10.8 (was 0.10.7) #1171 by @scala-steward
• metrics-core 4.1.12.1 (was 4.1.11) #1194 by @scala-steward
• sbt-java-formatter 0.6.0 (was 0.5.1) #1193 by @scala-steward
• jackson-databind 2.10.5 (was 2.10.4) #1169 by @scala-steward
• metrics-core 4.1.11 (was 4.1.10.1) #1172 by @scala-steward
• mockito-core 3.4.6 (was 2.28.2) #1173 by @scala-steward
• sbt-scalafmt 2.4.2 (was 2.4.0) #1174 by @scala-steward
• Update paradox akka to 0.35, new links to LB.com #1167 by @johanandren

Release notes in the Alpakka Kafka documentation

Alpakka Kafka core

• Emit offset batches on timer #1160 by @jyates
• Do not commit offsets for unassigned partitions #1123 by @jyates

Alpakka Kafka Akka Cluster Sharding

• Use bulk update API for shard allocation #1139 by @chbatey
• Don't use ctx.log in KafkaClusterSharding #1138 by @chbatey

Alpakka Kafka Testkit

• Deprecate embedded kafka support #1136 by @seglo
• Replace embedded kafka with schema registry with testcontainers #1135 by @seglo
• Add produce consume check to KafkaContainerCluster #1131 by @seglo

Updates

• mockito-core 2.28.2 (was 2.24.5) #1151 by @scala-steward
• scala-collection-compat 2.1.6 (was 2.1.2) #1152 by @scala-steward
• jul-to-slf4j, log4j-over-slf4j 1.7.30 (was 1.7.26) #1155 by @scala-steward
• embedded-kafka 2.4.1.1 (was 2.4.1) #1147 by @scala-steward
• protobuf-java 3.12.2 (was 3.11.4) #1142 by @scala-steward
• sbt 1.3.13 (was 1.3.9) #1153 by @scala-steward
• akka-stream-alpakka-csv 2.0.1 (was 2.0.0) #1143 by @scala-steward

Release notes in the Alpakka Kafka documentation

Alpakka Kafka core

• Committing: retry commits marked as retriable #1111 by @ennru
• Build: Jackson 2.10.4 and sbt plugin updates #1112 by @ennru
• Security: mask passwords in settings' toString #1110 by @ennru
• Commit when next offset is observed #1093 by @seglo
• Non-streaming producer wrapper #1085 by @ennru
• Simplify commit replying #1095 by @ennru
• DrainingControl: creation in toMat #1084 by @ennru
• Simpler use with the new actors API #1088 by @ennru
• Cluster-sharding: single listener per entity type key + Java DSL #1080 by @chbatey
• Drain partial offset commit batches on upstream failure #1058 by @sdudzin
• Kafka cluster sharding support with external sharding allocation #1067 by @seglo
• Add MetadataClient getCommittedOffsets #1073 by @epalace