Automatic data change tracking for PostgreSQL
Website · Docs · Report Bug · Request Feature · Discord · Twitter · LinkedIn
Bemi automatically tracks database changes ensuring 100% reliability and a comprehensive understanding of every change. It does it by connecting PostgreSQL's Write-Ahead Log (WAL) and implementing Change Data Capture (CDC) data pattern. Designed with simplicity and non-invasiveness in mind, Bemi operates in the background and doesn't require any alterations to your existing database tables.
There's a wide range of use cases that Bemi is built for! The tech was initially built as a compliance engineering system for fintech that supported $15B worth of assets under management, but has since been extracted into a general-purpose utility. Some use cases include:
You can install these system dependencies manually or use Devbox that relies on Nix Packages for providing isolated shells without containerization.
And of course, you need a PostgreSQL database that you want to connect to to track data changes. Make sure your database has SHOW wal_level;
returning logical
. Otherwise, you need to run the following SQL command and restart your PostgreSQL server:
ALTER SYSTEM SET wal_level = logical;
After installing all system dependencies, install all project dependencies with Node.js:
make worker-setup && cd worker && npm install
Alternatively, you can use Devbox instead and run a single command that will also install Node.js with pnpm and NATS server:
make worker-install
Set environment variables specifying connection settings for a PostgreSQL database you want to track:
export DB_HOST=127.0.0.1 DB_PORT=5432 DB_NAME=postgres DB_USER=postgres DB_PASSWORD=postgres
Run a worker as a single process with directly installed Node.js:
cd worker && npm concurrently -- "npm:up:*"
# Alternatively, with Devbox
make worker-up
Now try making some database changes like:
UPDATE _bemi_migrations SET executed_at = NOW() WHERE id = 1;
This will add a new record in the changes
table within the same database after a few seconds.
To optionally automatically enhance these low-level database changes with application-specific context (e.g., user ID, API endpoint, etc.), check out our compatible ORM packages.
Bemi consists of three main parts:
core
that we rely on for our Bemi cloud platform.The described architecture and the worker
code in this repository are a simplified version that can be easily run without much overhead.
If you want to self-host it in a production environment, see our self-hosting docs.
Alternatively, sign up with your GitHub login to use Bemi Cloud.
Distributed under the terms of the SSPL-1.0 License. If you need to modify and distribute the code, please release it to contribute back to the open-source community.