MQTT Proxy allows MQTT clients to send messages to other messaging systems
Work in progress
MQTT Proxy allows MQTT clients to send messages to other messaging systems
Download the latest Linux release
TAG=$(curl --silent "https://api.github.com/repos/grepplabs/mqtt-proxy/releases/latest" | jq -r '.tag_name')
curl -Ls https://github.com/grepplabs/mqtt-proxy/releases/download/${TAG}/mqtt-proxy-${TAG}-linux-amd64.tar.gz | tar xz
Move the binary in to your PATH.
sudo mv ./mqtt-proxy /usr/local/bin/mqtt-proxy
make clean build
make clean docker-build
Deploy the Helm chart
git clone [email protected]:grepplabs/mqtt-proxy.git
helm install mqtt-proxy ./mqtt-proxy/charts/mqtt-proxy \
--set image.tag=latest \
--set image.repository=grepplabs/mqtt-proxy \
--values <(echo '{
"extraArgs" : ["server","--mqtt.publisher.name=noop"]
}')
prerequisites
build and start-up test environment
cd scripts/cp-kafka
make build-up
subscribe to Kafka topic
docker exec -it broker kafka-console-consumer --bootstrap-server localhost:9092 --topic mqtt-test --property print.key=true --from-beginning
publish messages using mosquitto client
docker exec -it mqtt-client mosquitto_pub -L mqtt://mqtt-proxy:1883/dummy -m "test qos 0" --repeat 1 -q 0
docker exec -it mqtt-client mosquitto_pub -L mqtt://mqtt-proxy:1883/dummy -m "test qos 1" --repeat 1 -q 1
docker exec -it mqtt-client mosquitto_pub -L mqtt://mqtt-proxy:1883/dummy -m "test qos 2" --repeat 1 -q 2
docker exec -it mqtt-client mosquitto_pub -L mqtt://mqtt-proxy:1883/dummy -m "test qos 0 / v5" --repeat 1 -q 0 -V mqttv5
docker exec -it mqtt-client mosquitto_pub -L mqtt://mqtt-proxy:1883/dummy -m "test qos 1 / v5" --repeat 1 -q 1 -V mqttv5
docker exec -it mqtt-client mosquitto_pub -L mqtt://mqtt-proxy:1883/dummy -m "test qos 2 / v5" --repeat 1 -q 2 -V mqttv5
docker exec -it mqtt-client mosquitto_pub -L mqtt://mqtt-proxy-ssl:1884/dummy -m "test qos 0" --repeat 1 -q 0
docker exec -it mqtt-client mosquitto_pub -L mqtt://mqtt-proxy-ssl:1884/dummy -m "test qos 1" --repeat 1 -q 1
docker exec -it mqtt-client mosquitto_pub -L mqtt://mqtt-proxy-ssl:1884/dummy -m "test qos 2" --repeat 1 -q 2
check the prometheus metrics
watch -c 'curl -s localhost:9090/metrics | grep mqtt | egrep -v '^#''
provision test MSK and EC2 running in podman 2 proxy containers
cd scripts/msk
make tf-apply
create Kafka mqtt-test topic
publish
mosquitto_pub -m "on" -t "dummy" -k 20 -i mqtt-proxy.clientv --repeat 1 -q 1 -h <ec2-ip> -p 1883
mosquitto_pub -m "on" -t "dummy" -k 20 -i mqtt-proxy.clientv --repeat 1 -q 1 -h <ec2-ip> -p 1884
Create AWS SQS test1
and test2.fifo
queues
Build and start MQTT Proxy
make build
./mqtt-proxy server \
--mqtt.publisher.name=sqs \
--mqtt.publisher.message-format=json \
--mqtt.publisher.sqs.queue-mappings=test1='^dummy$' \
--mqtt.publisher.sqs.default-queue=test2.fifo \
--mqtt.publisher.sqs.aws-profile=admin-dev
publish
mosquitto_pub -L mqtt://localhost:1883/dummy -m "test qos 2" --repeat 1 -q 2
Create AWS SNS test1
and test2.fifo
topics with some subscription
Build and start MQTT Proxy
make build
./mqtt-proxy server \
--mqtt.publisher.name=sns \
--mqtt.publisher.message-format=json \
--mqtt.publisher.sns.topic-arn-mappings=arn:aws:sns:eu-central-1:123456789012:test1='^dummy$' \
--mqtt.publisher.sns.default-topic-arn=arn:aws:sns:eu-central-1:123456789012:test2.fifo \
--mqtt.publisher.sns.aws-profile=admin-dev
publish
mosquitto_pub -L mqtt://localhost:1883/dummy -m "test qos 2" --repeat 1 -q 2
Start rabbitmq and create test queue
cd scripts/rabbitmq
docker-compose up
curl -i -u user:bitnami -H "content-type:application/json" -XPUT -d'{"durable":true}' http://localhost:15672/api/queues/%2f/test
Build and start MQTT Proxy
make build
./mqtt-proxy server \
--mqtt.publisher.name=rabbitmq \
--mqtt.publisher.rabbitmq.username=user \
--mqtt.publisher.rabbitmq.password=bitnami \
--mqtt.publisher.rabbitmq.default-queue=test \
--mqtt.publisher.rabbitmq.confirms.exactly-once=true
publish
mosquitto_pub -L mqtt://localhost:1883/dummy -m "test qos 0" --repeat 1 -q 0
start server with plain
authenticator
cat <<EOF > mqtt-credentials.csv
alice,alice-secret
"bob","bob-secret"
EOF
mqtt-proxy server --mqtt.publisher.name=noop \
--mqtt.handler.auth.name=plain \
--mqtt.handler.auth.plain.credentials-file=mqtt-credentials.csv
mqtt-proxy server --mqtt.publisher.name=noop \
--mqtt.handler.auth.name=plain \
--mqtt.handler.auth.plain.credentials=alice=alice-secret \
--mqtt.handler.auth.plain.credentials=bob=bob-secret
publish
mosquitto_pub -m "on" -t "dummy" -u alice -P alice-secret
mosquitto_pub -L mqtt://bob:bob-secret@localhost:1883/dummy -m "on"
Kafka producer configuration properties used by librdkafka should be prefixed with producer.
--mqtt.publisher.kafka.config=producer.sasl.mechanisms=PLAIN,producer.security.protocol=SASL_SSL,producer.sasl.username=myuser,producer.sasl.password=mypasswd
mqtt-proxy server --mqtt.publisher.name=noop --mqtt.handler.ignore-unsupported SUBSCRIBE --mqtt.handler.ignore-unsupported UNSUBSCRIBE
metric | labels | description |
---|---|---|
mqtt_proxy_build_info | branch, goversion, revision, revision | A metric with a constant '1' value labeled by version, revision, branch, and goversion from which mqtt_proxy was built. |
mqtt_proxy_server_connections_active | Number of active TCP connections from clients to server. | |
mqtt_proxy_server_connections_total | Total number of TCP connections from clients to server. | |
mqtt_proxy_handler_requests_total | type, version | Total number of MQTT requests labeled by package control type and protocol version. |
mqtt_proxy_handler_responses_total | type, version | Total number of MQTT responses labeled by package control type and protocol version. |
mqtt_proxy_publisher_publish_duration_seconds | name, type, qos | Histogram tracking latencies for publish requests. |
mqtt_proxy_authenticator_login_duration_seconds | name, code, err | Histogram tracking latencies for login requests. |