equivalent to kafka-streams :octopus: for nodejs :sparkles::turtle::rocket::sparkles:
// suggested Node.js version: v12.16.1
npm install --save kafka-streams
const {KafkaStreams} = require("kafka-streams");
const config = require("./config.json");
const factory = new KafkaStreams(config);
const kstream = factory.getKStream("input-topic");
const ktable = factory.getKTable(/* .. */);
kstream.merge(ktable).filter(/* .. */).map(/* .. */).reduce(/* .. */).to("output-topic");
CHANGES: The latest version brings a lot of changes, please check here before updating.
>= 0.11.x
>= 8.x.x
If you are using the native mode (config: { noptions: {} }
).
You will have to manually install node-rdkafka
alongside kafka-streams.
(This requires a Node.js version between 9 and 12 and will not work with Node.js >= 13, last tested with 12.16.1)
On Mac OS High Sierra / Mojave:
CPPFLAGS=-I/usr/local/opt/openssl/include LDFLAGS=-L/usr/local/opt/openssl/lib yarn add --frozen-lockfile [email protected]
Otherwise:
yarn add --frozen-lockfile [email protected]
(Please also note: Doing this with npm does not work, it will remove your deps, npm i -g yarn
)
kafka-streams :octopus: equivalent for nodejs :sparkles::turtle::rocket::sparkles: build on super fast :fire: observables using most.js :metal:
ships with sinek :pray: for backpressure
comes with js and native Kafka client, for more performance and SSL, SASL and Kerberos features
the lib also comes with a few window
operations that are more similar to Apache Flink,
yet they still feel natural in this api :squirrel:
overwriteable local-storage solution allows for any kind of datastore e.g. RocksDB, Redis, Postgres..
async (Promises) and sync stream operators e.g. stream$.map()
or stream$.asyncMap()
super easy API :goberserk:
the lib is based on sinek
, which is based on kafka-node's ConsumerGroups
librdkafka
for more performanceYes.
Probably, yes. :smile:
Forks or Stars give motivation :bowtie: