Raccoon is a high-throughput, low-latency service to collect events in real-time from your web, mobile apps, and services using multiple network protocols.
Raccoon is high throughput, low-latency service that provides an API to ingest clickstream data from mobile apps, sites and publish it to Kafka. Raccoon uses the Websocket protocol for peer-to-peer communication and protobuf as the serialization format. It provides an event type agnostic API that accepts a batch (array) of events in protobuf format. Refer here for proto definition format that Raccoon accepts.
To know more, follow the detailed documentation
Raccoon can be used as an event collector, event distributor and as a forwarder of events generated from mobile/web/IoT front ends as it provides an high volume, high throughput, low latency event-agnostic APIs. Raccoon can serve the needs of data ingestion in near-real-time. Some domains where Raccoon could be used is listed below
Explore the following resources to get started with Raccoon:
Prerequisite
Run Docker Image
Raccoon provides Docker image as part of the release. Make sure you have Kafka running on your local and run the following.
# Download docker image from docker hub
$ docker pull raystack/raccoon
# Run the following docker command with minimal config.
$ docker run -p 8080:8080 \
-e SERVER_WEBSOCKET_PORT=8080 \
-e SERVER_WEBSOCKET_CONN_ID_HEADER=X-User-ID \
-e PUBLISHER_KAFKA_CLIENT_BOOTSTRAP_SERVERS=host.docker.internal:9093 \
-e EVENT_DISTRIBUTION_PUBLISHER_PATTERN=clickstream-%s-log \
raystack/raccoon
Run Docker Compose
You can also use docker-compose
on this repo. The docker-compose
provides raccoon along with Kafka setup. Then, run the following command.
# Run raccoon along with kafka setup
$ make docker-run
# Stop the docker compose
$ make docker-stop
You can consume the published events from the host machine by using localhost:9094
as kafka broker server. Mind the topic routing when you consume the events.
Prerequisite:
# Clone the repo
$ git clone https://github.com/raystack/raccoon.git
# Build the executable
$ make
# Configure env variables
$ vim .env
# Run Raccoon
$ ./out/raccoon
Note: Read the detail of each configurations here.
# Running unit tests
$ make test
# Running integration tests
$ cp .env.test .env
$ make docker-run
$ INTEGTEST_BOOTSTRAP_SERVER=localhost:9094 INTEGTEST_HOST=localhost:8080 INTEGTEST_TOPIC_FORMAT="clickstream-%s-log" GRPC_SERVER_ADDR="localhost:8081" go test ./integration -v
Development of Raccoon happens in the open on GitHub, and we are grateful to the community for contributing bugfixes and improvements. Read below to learn how you can take part in improving Raccoon.
Read our contributing guide to learn about our development process, how to propose bugfixes and improvements, and how to build and test your changes to Raccoon.
To help you get your feet wet and get you familiar with our contribution process, we have a list of good first issues that contain bugs which have a relatively limited scope. This is a great place to get started.
This project exists thanks to all the contributors.
Raccoon is Apache 2.0 licensed.