Distributed、High availability、Simplicity、Visualized. Base on mysql binlog , process binlog into understandable message and send to kafka etc
Binlogo is the distributed, visualized application based on MySQL binlog. In short, binlogo is to process the data changes of MySQL into easily understand messages and output to different systems according to the user's configuration. Here are part of advantages:
Install etcd. Binlogo relies on etcd, so you must install etcd first.
Install binlogo. Binlogo's download address: Download Address
Message Format: Data format of binlogo output
Start binlogo.
Edit config. ${binlogo-path}/etc/binlogo.yaml
$ ./binlogo server --config ./etc/binlogo.yaml
Open up your browser to http://127.0.0.1:9999/console to view the console website
Create Pipeline: config.go
Follow the steps.
Click button to run the pipeline instance.
You can see the operation condition of pipeline.
Insert some into mysql, watch the ouput on stdout.
- High performance, possible data loss.
- acks=1
- enable.idempotence=false
- compression.type=snappy
- retries=0
- For reliability performance:
- acks=-1
- enable.idempotence=true
- retries=3 or larger one
$ docker pull jin06/binlogo $ docker run -e "ETCD_ENDPOINTS=172.17.0.3:2379" --name BinlogoNode -it -d -p 9999:9999 jin06/binlogo:latest Open browser access http://127.0.0.1:9999/console I started five nodes with docker. The following is a screenshot