• Add default support for MSSQL auto-incrementing bigint and smallint (#1435, @justinfalk)
• Improve pgloader docs (#1440, @dimitri)
• Update test dependencies for Debian (@df7cb)

• Upgrade Clozure-CL in the Dockerfile.ccl (@dimitri)
• Use the unix-namestring as the hash key for SQL queries (#1420) (@dimitri)
• Added support for sequences with minvalue defined (#1429) (@noctarius)
• Fix mapping mysql signed int with auto_incement to postgresql serial (#1248, #1437) (@padinko)
• Debian packaging: Depend on libsqlite3-0 (@df7cb)

• Set SBCL dynamic space size to 16 GB on 64 bit architectures.
• Improve documentation with command lines and defaults.
• SBCL compiler notes should not be fatal to pgloader.

• Maintenance release fixing the bundle build process

• Run testsuite from github action and Debian autopkgtest (@df7cb)
• Fix looping over sbcl external-formats (@dimitri)
• Force libcrypto reload in src/hooks.lisp (@athos-ribeiro)
• Fix minor casing issue in intro docs (@JockeTF)
• Allow underscores in cast type names (@swt30)
• Specify v8 of freetds (@dagostinelli)
• Parameterize DYNSIZE in dockerfiles (@BrendanBall)
• Fix documentation typo (@Nedeas)

Fix dependencies with the latest/current libs (including esrap and cl-postmodern). Work done by the debian packaging maintenance team, kudos folks!

Bugfix release that fixes building pgloader with SBCL 2.0.1 and newer (GitHub issue #1087) due to CFFI 0.20.0 not being compatible with the new SBCL releases.

From other noteworthy things, the DBF format received a lot of improvements.

This release contains three major themes: usual maintenance and bug fixing, support for new database systems as sources and targets, and support for Citus distribution.

New Documentation System

The documentation also has received quite some attention and is now available at https://pgloader.readthedocs.io/en/latest/. It includes a new introduction page and the introduction section now has a Feature Matrix displaying what feature coverage you can expect depending on your database source type. Please read https://pgloader.readthedocs.io/en/latest/intro.html for more details.

PostgreSQL as a source

pgloader v3.6.1 now as integrated support for PostgreSQL either as a source database system, or a target database system, or both. Migrating from PostgreSQL to another PostgreSQL instance is best done with PostgreSQL tools such as logical replication or backup and restore facilities. That said, pgloader also supports PostgreSQL derivatives such as Redshift and Citus, so from PostgreSQL to PostgreSQL is to be read with that in mind.

Support for Citus distribution keys

pgloader v3.6.1 includes support for Citus distribution, documented at https://pgloader.readthedocs.io/en/latest/ref/pgsql-citus-target.html. When your target database is Citus, then pgloader has support for:

• distribution key declaration right in the pgloader command, allowing for the next items,
• distribution key integration in the target schema, done on the flight by following foreign key definitions and adding the distribution key where needed (tables and constraints),
• backfilling of the data from their sources, using SQL JOINs when migrating the data from the source to the target system.

Support for Redshift

pgloader v3.6.1 implements Redshift support both as a source and as a target database system. When used as a target, pgloader takes care of dumbing down the data types when compared to PostgreSQL, and needs the user to provide an S3 setup where to upload intermediary files: Redshift can COPY from S3 files, not from standard input on the connection like PostgreSQL would.

When used as a source system, pgloader uses SELECT queries with Redshift, allowing to fetch all the data over the same network protocol, as usual.

AFTER SCHEMA EXECUTE SQL

pgloader v3.6.1 implements new support for running SQL queries in between its handling of the schema and the data parts of the migration. This allows for custom post-processing the schema to happen before loading the data into the target database.

Improvements and bug fixes

The existing support for MS SQL, MySQL, SQLite, CSV and other formats have received improvements and bug fixes. The CSV parser for instance is now able to consider a subset of the target table columns when using a CSV header in the file, or column and fields in different orders.

Sponsoring pgloader

Some of the improvements in that release were made possible thanks to our sponsors! If you need new pgloader features, please talk to me about them, I'll be happy to make it happen! You can contribute to pgloader either your time and skills, or money. See https://pgloader.io/moral-licence/ for more information and details.

This release is meant to allow for more testing of the current pgloader code, which is known to fix the most recent issues on docker and debian derived systems, and some compilation errors with outdated dependencies.

Please report oddities!

This release is a maintenance release that contains a new feature, because it happened that way:

• debian package fixes, thanks to Christoph Berg and Sébastien Villemot!
• usual set of bug fixes, see the issues of the project for more information, or the commit logs,
• add support for Redshift as a target database, using S3 as intermediary files.

The main driver behind that release is the debian packaging fixes!