Standalone Karafka library for producing Kafka messages
WaterDrop is a standalone gem that sends messages to Kafka easily with an extra validation layer. It is a part of the Karafka ecosystem.
It:
1.0+
and Ruby 2.7+
Karafka ecosystem components documentation, including WaterDrop, can be found here.
If you want to both produce and consume messages, please use Karafka. It integrates WaterDrop automatically.
To get started with WaterDrop:
bundle add waterdrop
producer = WaterDrop::Producer.new do |config|
config.deliver = true
config.kafka = {
'bootstrap.servers': 'localhost:9092',
'request.required.acks': 1
}
end
# sync producing
producer.produce_sync(topic: 'my-topic', payload: 'my message')
# or for async
producer.produce_async(topic: 'my-topic', payload: 'my message')
# or in sync batches
producer.produce_many_sync(
[
{ topic: 'my-topic', payload: 'my message'},
{ topic: 'my-topic', payload: 'my message'}
]
)
# and async batches
producer.produce_many_async(
[
{ topic: 'my-topic', payload: 'my message'},
{ topic: 'my-topic', payload: 'my message'}
]
)
# transactions
producer.transaction do
producer.produce_async(topic: 'my-topic', payload: 'my message')
producer.produce_async(topic: 'my-topic', payload: 'my message')
end