Azure Event Hubs Spark Versions Save

Release notes:

• performance improvement - added data cache to avoid unnecessary fetch operations for multiple readers scenario (#608 )

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.21/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.21/jar

Release notes:

• Update java SDK dependency which contains a fix for race conditions resulting in timeout error. (#604)

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.20/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.20/jar

Release notes:

• Support parameters for Azure AD authentication (#596)
• Improve logging (#575, #600)
• Fix bugs (#584, #577)

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.19/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.19/jar

Release notes:

• Support Azure AD authentication (#535)
• Support partition scale-out scenario (#500, #544)
• Fix bugs (#550, #505)
• ThrottlingStatus plug-in update (#555)

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.18/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.18/jar

Release notes:

• Spark runtime 3.0 support (#520).
• Slow partition adjustment support (#512).
• Fix an issue of retrying receiver close (#523).

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.17/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.17/jar

Release notes:

• Fix added quotes to string property values (#516).
• Add receive watchdog (#517).
• EventHubs write - allow Properties map to be non-nullable or nullable column (#507).
• Retry on send failures, improve logging and metrics (#504)
• Support Apache Kafka key property (#501)

For PySpark scenario, version 2.3.15+ assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.16/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.16/jar

Release notes:

• Add support for different partitioning strategies to deal with reader/executor skew (#478).
• Add null check for properties key-value and a test case (#483)
• Fix an issue of closing a receiver for epoch receiver case (#486)
• Fix IllegalStateException issue and recreate a receiver if the receiver was closed (#490)
• Encrypt connection string in eventhub conf (#491)
• Use receiver timeout for underlying API calls to Event Hubs so that connector can retry on failures (#492)
• Add receive, send listener for getting performance metrics (#473)
• Fix a few issues for Scala 2.12 and code clean-up (#498)
• Add SimpleLogMetricPlugin class (#499)

For PySpark scenario, version 2.3.15 assumes that connection string in the config dictionary is encrypted by the encrypt API in the EventHubsUtils class.

Package link (Scala 2.11) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.15/jar

Package link (Scala 2.12) - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.12/2.3.15/jar

Release notes:

• add event hub name to the OffsetRange toString function so that it is present in the spark metadata. add simple unit test for validation of the string (#450).

Package link - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.13/jar

Release notes:

• Update Azure Event Hubs SDK dependency and use the latest version of the SDK that contains a fix for the issue of reconnect failure after a connection is terminated.

Package link - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.12/jar

Release notes:

• performance improvement for EventHubs writer (#441)

Package link - https://search.maven.org/artifact/com.microsoft.azure/azure-eventhubs-spark_2.11/2.3.11/jar