RPJiOS: RPJ's RPi OS, a sensor data platform for the Raspberry Pi built with python2.7 and redis.
A pub/sub-based implementation of a Raspberry Pi data pipeline built on redis and centered around sensors.
The general philosophy is that a "sensor" is any entity (physical or not) that operates primarily in an output-only mode (configuration doesn't necessarily count as an "input" so is allowable).
These outputs are treated ephemerally, in a "fire and forget" manner, creating a data stream. The intent is for interested entities to subscribe to the data stream and transform, interpret and/or persist it according to their requirements.
This is still very much an active work-in-progress! However, as it is functional and actively deployed I figured it was worth making public in the event it might help others in their projects.
Those current deployments consist of my atmospheric particulate matter sensor and my garden monitoring bots, with the entire system handling (on average) about a half-million units of sensor data per day.
raspi-config
):
cd
into repo dir./setup.sh
(you might need to enter your sudo
password to install requirements)source env/bin/activate
and go!-r 1
)-o
to the same as -i
)-t
to key
)-m
to flatten:[options]
)-t
to list:[options]
, where the limit=X
option sets the bound)downsample -i redis://localhost -o redis://sql-db-host -r ... -p ...
sql-db-host
":
sqlite-sink path-to-db.sqlite3
The following are currently supported (with the required drivers / interfaces setup of course):
Variants of these sensors could likely be made to work with this system via simple modifications if any are required at all.
config.json
rpjctrl
, and I'm still using ledCount.py
on some of my units because I'm too lazy to re-write it properly