DBLab enables 🖖 database branching and ⚡️ thin cloning for any Postgres database and empowers DB testing in CI/CD. This optimizes database-related costs while improving time-to-market and software quality. Follow to stay updated.
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Any feedback will be greatly appreciated:
If you want to help the project grow, consider various ways of contributing:
Diff between versions v3.4.0 and v3.5.0: https://github.com/postgres-ai/database-lab-engine/compare/v3.4.0...v3.5.0
demo-token
)We greatly value your feedback. Connect with us via:
Additional resources where you can get insights about DBLab and Postgres:
Interested in giving back to the project? Here's how you can make an impact:
The new name for the Database Lab Engine is "DBLab Engine". Updates are currently underway across our materials to reflect this change. To align with this change, we have introduced specific domains for the product: dblab.dev
and dblab.sh
. For ease of access, we have established the following short URLs:
curl -sSL dblab.sh | bash
)demo-token
)demo-token
):bow: Many thanks to our first-time contributors: @mbobin, @TheLiamGuy, @gromsterus, @ibash, @byx01, @davidsvenson.
Thank you for the support!
We've expanded the installation options for DBLab SE. Besides its presence in the AWS Marketplace, you can now seamlessly install DBLab SE directly from the Postgres.ai Console.
This setup is entirely automated and can be used anywhere:
Check out the updated tutorial.
For those considering an upgrade, we recommend executing a fresh installation of DBLab. At present, we don't have automation in place for upgrades. If you need to update an existing DBLab installation, it's essential to stop and remove the current containers. Then, initiate a fresh start of the DBLab container using the 3.4.0
tag. More details can be found in the docs. Before initiation, ensure you've adjusted the DBLab configuration as outlined below.
There is a new method for running clones with customized port and socket directories. Please note that this is NOT backward compatible with previous versions of the Postgres images that DBLab utilized. It is crucial to ensure the dockerImage
options utilize tags with the -0.3.0
suffix or later, such as postgresai/extended-postgres:15-0.3.0
(details can be found in this MR).
The global.telemetry
subsection has been deprecated and removed. The control to activate/deactivate telemetry has been relocated to the platform
section:
platform:
url: "https://postgres.ai/api/general"
enableTelemetry: true
From version v3.4.0
onward, the platform
section is enabled by default.
If either the accessToken
or orgKey
keys aren't empty, ensure the url
parameter is specified; otherwise, the DBLab instance will not initialize. By default, neither accessToken
nor orgKey
are defined when in case of self-managed installations.
platform:
url: "https://postgres.ai/api/general"
enableTelemetry: true
# orgKey: "org_key"
# accessToken: "platform_access_token"
Please omit the following Docker mount directives during container startup, as they are no longer necessary:
--volume /sys/kernel/debug:/sys/kernel/debug:rw \
--volume /lib/modules:/lib/modules:ro \
--volume /proc:/host_proc:ro \
ignoreErrors
in subsections logicalDump
and logicalRestore
to allow not to interrupt the process of dump/restore in case of errors - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/699, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/700, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/730
skipPolicies
in subsection logicalRestore
to allow to skip policies (CREATE POLICY
) during restore process, to avoid errors like policy "xxx" for table "yyy" already exists
: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/769
cloneAccessAddresses
in section provision
to control port publishing for clone containers; now limited to 127.0.0.1
by default; support multiple addresses and IPv6: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/786
9.6-0.3.0
... 15-0.3.0
; :warning: older tags won't work with DBLab 3.4.xpgvector
extension added to/status
with proper messages - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/696
portPool
inclusive (previously, it was mistakenly exclusive, leading to unexpected behavior): https://gitlab.com/postgres-ai/database-lab/-/merge_requests/785
Error response from daemon: Duplicate mount point: /var/lib/dblab/dblab_pool/dataset_1/dump/
: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/791
trusted.gpg.d
because apt-key
is deprecated - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/701
Diff between versions v3.3.0 and v3.4.0: https://github.com/postgres-ai/database-lab-engine/compare/v3.3.0...v3.4.0
demo-token
)We greatly value your feedback. Connect with us via:
Additional resources where you can get insights about DBLab and Postgres:
Interested in giving back to the project? Here's how you can make an impact:
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Any feedback will be greatly appreciated:
If you want to help the project grow, consider various ways of contributing:
Diff between versions v3.2.0 and v3.3.0: https://github.com/postgres-ai/database-lab-engine/compare/v3.2.0...v3.3.0
Release v3.2.0 delivers a good number of new features and improvements. DLE UI is extended with two new tabs, "Logs" and "Configuration", to improve the user experience when configuring or reconfiguring DLE for managed Postgres services such as AWS RDS and Google Cloud SQL:
This allows users to see PGDATA initialization errors when setting up DLE, then adjust the configuration and retry initialization right in the UI – without any need to connect to the DLE machine using SSH.
The demo DLE instance can be found here: https://demo.aws.postgres.ai:446/instance (token: demo_token
).
PostgreSQL 15 is already supported by DLE and included in the list of default images, with many extensions shipped with it: https://postgres.ai/docs/database-lab/supported-databases.
DLE in AWS Marketplace has now General Availability (GA) status, it has already been updated to version 3.2.0: https://aws.amazon.com/marketplace/pp/prodview-wlmm2satykuec, offering the Standard license, with support, embedded monitoring, simplified installation, and access to a wider variety of supported Postgres images (including extensions not available by default). An instance with DLE delivers "dev/test databases as a service" and levels up all dev&test processes. For example, on an r5.xlarge instance (4 vCPUs, 32 GiB RAM) with 200 GiB disk space, it is possible to run up to 30 database clones simultaneously (and even more, if shared_buffers
is adjusted to lower values) spending as low as $360 per month – less than for a single regular RDS clone of the same size.
:bow: The development team greets a new contributor: @Adrinlol. This release wouldn't be possible without other contributors: @agneum, @vitabaks, @DmitryFomin1, @ane4ka, @NikolayS, @denis256, @yuridevx.
queryPreprocessing
in logicalRestore
!576
queryPreprocessing
(available in sections logicalRestore
, logicalSnapshot
, physicalSnapshot
) !576
excludeTables
– allow excluding certain tables for data retrieval in logical mode: !536, https://gitlab.com/postgres-ai/database-lab/-/issues/348 (was also backpatched to 3.1.2)maintenance_work_mem
for Logical mode: !567
Diff between versions v3.1.2 and v3.2.0: https://github.com/postgres-ai/database-lab-engine/compare/v3.1.2...v3.2.0.
There are no special actions that are required to migrate from DLE 3.1.x to 3.2.0: you need to stop the DLE container, change the image, and start a new one using the new image. All snapshots should remain available, and if, at stopping time, some clones were present, they should be automatically recreated by DLE.
Note: if you have an older version, check the previous release notes as well: https://gitlab.com/postgres-ai/database-lab/-/releases.
The config directory, by default, is ~/.dblab/engine/configs
and the corresponding mounted volume must be writable:
...
--volume ~/.dblab/engine/configs:/home/dblab/configs
...
Make sure that ~/.dblab/engine/logs
is mounted as a volume to the DLE container:
...
--volume ~/.dblab/engine/configs:/home/dblab/configs \
--volume ~/.dblab/engine/meta:/home/dblab/meta \
--volume ~/.dblab/engine/logs:/home/dblab/logs \
...
Alternatively, you can mount the entire directory (including configs
and meta
– but make sure that these subdirectories exist, DLE doesn't create them yet):
...
--volume ~/.dblab/engine:/home/dblab
...
The queryPreprocessing
section supports inline SQL now. Examples can be found here: !576#examples
Because the configuration can be viewed and edited in the UI, make sure:
verificationToken
, Platform accessToken
, you can find all masking rules here: !608#masking-rules)To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Any feedback will be greatly appreciated:
If you want to help the project grow, consider various ways of contributing:
DLE v3.1.2 delivers multiple bug fixes and improvements.
:bow: The development team greets new contributors: @yuridevx, @denis256.
excludeTables
– exclude certain tables for data retrieval in logical mode: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/536, https://gitlab.com/postgres-ai/database-lab/-/issues/348
--restricted
flag https://gitlab.com/postgres-ai/database-lab/-/merge_requests/532, https://gitlab.com/postgres-ai/database-lab/-/issues/356
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Any feedback will be greatly appreciated:
If you want to help the project grow, consider various ways of contributing:
Diff between versions v3.1.1 and v3.1.2: https://github.com/postgres-ai/database-lab-engine/compare/v3.1.1...v3.1.2
DLE v3.1.1 delivers multiple bug fixes and improvements.
The development team greets a new contributor: @Alexand.
shared_preload_libraries
- https://gitlab.com/postgres-ai/database-lab/-/merge_requests/522
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Any feedback will be greatly appreciated:
If you want to help the project grow, consider various ways of contributing:
Diff between versions v3.1.0 and v3.1.1: https://github.com/postgres-ai/database-lab-engine/compare/v3.1.0...v3.1.1
DLE 3.1 has many improvements and several new features, such as native support of pgBackRest and the ability to configure the timezone when using CLI.
In this release, the development team has focused on the Database Lab Engine community, making it easier to get help or contribute. The team greets all new contributors: @Nikolay Devxx, @asotolongo, @Tanya301, @denis-boost, @pietervincken, @ane4ka
The README.md has been significantly improved. It now has four translations, with more coming in the future. Many thanks to our README translation contributors! It helps engineers worldwide understand DLE concepts and improve the development experience by building powerful dev&test environments.
:warning: Prior to version 3.1, the default ZFS version DLE's Docker images were using was 0.8.x. For DLE 3.1, the default Docker image (postgresai/dblab-server:3.1.0
) uses ZFS 2.1.2. If you need to upgrade DLE from a previous version that worked with a ZFS pool 0.8.x (default for previous DLE versions), consider the following options:
zpool upgrade dblab_pool
), and then upgrade DLE to use the default image postgresai/dblab-server:3.1.0
postgresai/dblab-server:3.1.0-zfs0.8
/snapshot
API calls which could take many seconds in the past https://gitlab.com/postgres-ai/database-lab/-/merge_requests/426
dataStateAt
is determined - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/356
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Any feedback will be greatly appreciated:
If you want to help the project grow, consider various ways of contributing:
Diff between versions v3.0.0 and v3.1.0-rc.1: https://github.com/postgres-ai/database-lab-engine/compare/v3.0.0...v3.1.0-rc.1
DLE 3.0.3 is a backward-compatibility update to ensure that the DLE image is compatible with existing datasets (ZFS 0.8.*). In DLE 3.0.2, in the Docker image ZFS version was prematurely switched to 2.*. The release 3.0.3 mitigates it. There are plans to switch to ZFS 2.* in the future major versions of DLE.
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository on GitLab or GitHub:
Make sure the DLE running command contains the environment variable --env DOCKER_API_VERSION="1.41"
.
The maximum supported Docker API version is 1.41, but the client version (1.42) is too new.
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Any feedback would be greatly appreciated:
Diff between versions v3.0.2 and v3.0.3: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.2...v3.0.3
DLE 3.0.2 is a security update, mitigating multiple CVEs reported by various security inspection tools. There are no known critical vulnerabilities that are fixed or are still unfixed in this release but upgrading at the earliest convenience is recommended.
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository on GitLab or GitHub:
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Any feedback would be greatly appreciated:
Diff between versions v3.0.1 and v3.0.2: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.1...v3.0.2
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository: https://gitlab.com/postgres-ai/database-lab.
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Any feedback would be greatly appreciated:
Diff between versions v3.0.0 and v3.0.1: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.0...v3.0.1