Build Postgres Extensions with Rust!
Welcome to pgrx v0.11.4!
This is a minor release to add some additional functionality to cargo-pgrx
to enable running tests in atypical CI configurations and the ability to turn off its colored output via the NO_COLOR
environment variable. This minor release also includes a small change that should measurably improve compilation times for all extensions.
To update to this this, please run cargo install cargo-pgrx --version 0.11.4 --locked
and ensure to update your extension's Cargo.toml
file to use pgrx v0.11.4.
cargo-pgrx
changesPR #1652 was cherry-picked into this release to include the new --runas
and --pgdata
arguments to cargo pgrx test
. --runas
uses the system sudo
command to run the temporary Postgres cluster process as the specified user, and uses --pgdata
as the location where the cluster's PGDATA directory should be initialized.
cargo-pgrx
now understands the NO_COLOR
and FORCE_COLOR
environment variables supported by the upstream dependency it uses for generated colored output. See https://crates.io/crates/owo-colors for details. This is from PR #1664
pgrx generates a (very large) set of Rust bindings that represent the "rust version" of all the internal Postgres symbols (functions, globals, etc). In the case of functions, pgrx also generates wrapper functions so it can instrument calls with the proper guard rails. It turns out, simply annotating these wrapper functions with #[inline]
drastically improves compilation time as it allows the rust compiler to defer their compilation until the very end. Which ultimately means only the functions used by an extension get compiled -- the others are DCE'd. This came from PR #1616.
Thanks to @usamoi for improving our compilation times, and shout out to @theory for his first contribution!
Full Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.11.3...v0.11.4
Welcome to pgrx 0.12.0-alpha.1!
Say the magic words with me!
cargo install cargo-pgrx --locked --version 0.12.0-alpha.1
Perhaps the most exciting change this round is @usamoi's contribution in https://github.com/pgcentralfoundation/pgrx/pull/1468 which means that we no longer perform a dlopen
in order to generate the schema. The cost, such as it is, is that your pgrx extensions now require a src/bin/pgrx_embed.rs
, which will be used to generate the schema. This has much less cross-platform issues and will enable supporting things like cargo binstall
down the line.
It may be a bit touchy on first-time setup for transitioning older repos. If necessary, you may have to directly add a src/bin/pgrx_embed.rs
and add the following code (which should be the only code in the file, though you can add comments if you like?):
::pgrx::pgrx_embed!();
Your Cargo.toml will also want to update its crate-type key for the library:
[lib]
crate-type = ["cdylib", "lib"]
ManuallyDropUnion
thanks to @NotGyro in https://github.com/pgcentralfoundation/pgrx/pull/1547
ErrorReport::report
is now unwrap_or_report
! https://github.com/pgcentralfoundation/pgrx/pull/1568
const
s are no longer fn
, thanks to @workingjubilee in https://github.com/pgcentralfoundation/pgrx/pull/1584
Interval::is_finite
since https://github.com/pgcentralfoundation/pgrx/pull/1594
*_tree_walker
functions to the same signature their *_impl
version in Postgres 16 has: https://github.com/pgcentralfoundation/pgrx/pull/1596
pg_sql_graph_magic!()
macro, which should help with more things in the future!We have quite a lot of useful additions to our API:
SpiClient::prepare_mut
was added thanks to @XeniaLu in https://github.com/pgcentralfoundation/pgrx/pull/1275
#[pg_test]
, you have been able to use #[should_panic(expected = "string")]
to anticipate a panic that contains that string in that test. For various reasons, #[pg_test(error = "string")]
is much the same. Now, you can also use #[pg_test(expected = "string")]
, in the hopes that is easier to stumble across, as of https://github.com/pgcentralfoundation/pgrx/pull/1570
Result<composite_type!("..."), E>
supportResult<composite_type!("Name"), E>
, as a case that had not been handled before.Thanks to @rjuju, @NotGyro, and @workingjubilee, we now have significantly expanded docs for cargo-pgrx and pgrx in general. Some of these are in the API docs on https://docs.rs or the READMEs, but there's also a guide, now! It's not currently published, but is available as an mdbook in the repo.
Some diagnostic information that is also arguably documentation, like comments and the suggestion to cargo install
, have also been improved, thanks to @workingjubilee in
#[pg_cast]
An experimental macro for a CREATE CAST
was contributed by @xwkuang5 in https://github.com/pgcentralfoundation/pgrx/pull/1445!
Thanks to @the-kenny in https://github.com/pgcentralfoundation/pgrx/pull/1490 and @workingjubilee in https://github.com/pgcentralfoundation/pgrx/pull/1504, it was brought to our attention that some dependencies had unusual legal requirements. So we fixed this with CI! We now check our code included into pgrx-using binaries is MIT/Apache 2.0 licensed, as is common across crates.io, using cargo deny
!. The build tools will have more flexible legal requirements (partly due to the use of Mozilla Public License code in rustls).
Many internal cleanups were done thanks to
In particular:
pfree
our Array
s we detoasted as-of https://github.com/pgcentralfoundation/pgrx/pull/1571
RawArray
is now low-overhead due to https://github.com/pgcentralfoundation/pgrx/pull/1587
We had a number of soundness issues uncovered or have added more tests to catch them.
&
and &mut
in fcinfo.rs
by @workingjubilee in https://github.com/pgcentralfoundation/pgrx/pull/1595
Part of the cleanup by @workingjubilee was reducing the number of deps we compile:
Hopefully it will reduce compile time and disk usage!
Full Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.12.0-alpha.0...v0.12.0-alpha.1
Hello and welcome to The Beginning of The End!
You may have seen some largely-inscrutable rambling about "object lifetimes" and "memory contexts" from me in various comments, logs, and issues. You may have also heard that this has effectively stalled all forward feature work. You may have noticed this has been the case for months now.
The long and short of it is that pgrx never actually fully built-in the actual lifetimes of objects in Postgres into its various types. As more and more interlocking parts grew, between things like the type unboxing in trait FromDatum
, or the bounded memory context of Spi::connect
, eventually many supposedly-safe interfaces arose which proved wildly unsafe in reality due to this mesh of interactions. Internal portions of pgrx relied on expeditious hacks to ignore the growing problems, until this rotten foundation yielded an enormous crop of soundness issues over the year of 2023. Even when not exploitable, they prevent otherwise-trivial enhancements in terms of speed or memory usage.
Key elements of a new foundation have landed in pgrx, which allow ascribing lifetimes to every datum and every memory context that a Postgres extension will encounter. There is much more work to be done to actually make it so all Rust code has to flow through these new interfaces. This alpha release is an extremely "preview" release to allow assessing some of the damage from the first set of breaking changes, so that some initial feedback and triage can occur. There will, unfortunately, be more breaking changes on the horizon.
There are so many small changes that I am simply going to summarize the main point of them:
If your extension now breaks in some inscrutable way that you cannot otherwise discern, the problem is probably hidden in macro expansion. This cannot really be diagnosed except using cargo expand
diffed against previous expansions. A most likely issue is that something that previously hid a lifetime during expansion or writes it as 'static now propagates it by name or as the anonymous '_ lifetime.
Datum<'src>
There are now lifetime-bound Datums. Expect to see them in more places.
UnboxDatum<As<'src> = Type<'src>>
Expect to see this or a similar signature a lot: there are new conversion traits which use a generic associated type or similar sneaky tactics to allow "clamping down" the lifetime to a bound, unless the produced type is always 'static by being a simple value like i32
.
&'mcx MemCx<'_>
You will see more and more references to this type, which is a memory context that encodes an inner invariant lifetime, in signatures. If one of these is currently in-scope, then please remember it is always unsafe
to manipulate the static mut CurrentMemoryContext
in Postgres, but it is doubly so then. In general, when designing your own types using pgrx, you should prefer designing code that uses a signature like that of MemoryContextAlloc
instead of palloc
if you are at all uncertain.
Full Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.11.0...v0.12.0-alpha.0
The pgrx 0.11.3 release addresses several UB risks of pgrx, updates its dependencies to address possible security risks on a few points, and includes considerable numbers of additional headers in the release. It should also now be easier to actually use cargo-pgrx on a mildly more interesting network configuration, though you may need to specify --no-default-features
with either --features native-tls
or --features rustls
when you cargo install cargo-pgrx
.
New bindings added thanks to
Two UB fixes!
ereport!
macro in https://github.com/pgcentralfoundation/pgrx/pull/1472
Together these should mean it's possible to actually use cargo-pgrx on whatever your network configuration is, but you might have to use cargo install --no-default-features --features native-tls
to install with native-tls (which, on Linux, means OpenSSL). By default, you will use rustls.
These address some largely-hypothetical security risks, but one is particularly important: the bindgen update means we now should be compatible with some aarch64 builds that might have failed.
Full Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.11.2...v0.11.3
pgrx v0.11.2 is a minor release which
Oid::from_u32_unchecked
API... because it is actually possible to do via casting in a query, which makes it effectively impossible for us to not wind up providing it via some other safe API, since the source is also an easily-Copied type. Thus you can now simply use From::from
for it. Thanks to @thomcc for this discovery, implemented by @workingjubilee in https://github.com/pgcentralfoundation/pgrx/pull/1374
As usual, cargo install cargo-pgrx --version 0.11.2 --locked
and be on your merry way!
Thanks everyone!
Full Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.11.1...v0.11.2
Hello. Welcome to pgrx v0.11.1, a bugfix release that hopefully addresses a very annoying persistent problem for users, especially ones that were trying to use pgrx with Postgres 16! My (@workingjubilee)'s changes in b992f55eac931c49f72f142741fac7f2922b078c now steer bindgen towards including the exact compiler-header directory that we require, instead of relying on clang-sys to guess the right directory in ways that can go horribly wrong. Unfortunately, this may make our build step more brittle in certain cases. Please report issues as they arise, so that we can continue to refine our build system, and eventually fix this in bindgen proper!
This new behavior can still be disabled by setting PGRX_BINDGEN_NO_DETECT_INCLUDES
, as it is still a form of autodetecting includes, but if this release fixes your build that was previously relying on that or any other easily-removed hack, please let us know!
Also, thanks to @LuminousMonkey, you can now use pgrx on illumos, an operating system descended from Solaris, with no changes to your source code!
Full Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.11.0...v0.11.1
Welcome to pgrx v0.11.0. It's a new "major" release with some new features, a faster "build.rs", and some bugfixes.
As always, please update by running cargo install cargo-pgrx --version 0.11.0 --locked
and update your extension crate Cargo.toml
files accordingly.
@workingjubilee did some work to improve build times. As measured on their computer against cargo check
, it's roughly a quarter faster. In doing so, the generated "bindings.rs" files look a little different and our "build.rs" has been cleaned up a bit.
is_for_release
once by @workingjubilee in https://github.com/pgcentralfoundation/pgrx/pull/1315
PgList
@workingjubilee did a tremendous amount of work creating a proper wrapper around Postgres' internal List
type, called PgList
.
Some background here is that we intend to add the "dynamic function call" work (see below) to PL/Rust. In doing that work, use of Postgres' List
type is pretty important and pgrx' existing implementation was a) mostly in the cshim, and b) just not very-good-and-sound. Considering (b) and also that PL/Rust isn't able to use anything from pgrx' chsim (a), @workingjubilee set out to make a proper Rust wrapper/implementation of Postgres' List
type.
Note: As part of this, NodeTag
is now a proper enum
, which requires a small migration which is mostly satisfied by a regex-based find-replace of NodeTag_T_[A-Za-z]*
with NodeTag::T_$1
.
We've added more Postgres internals headers to the pgrx-pg-sys
crate:
pgrx now allows calling any SQL-defined function (ie, via CREATE FUNCTION
) in a (mostly) Rusty way with the new dynamic function calling support. Some documentation around this feature can be found in the sources, here and in the unit tests.
As mentioned above, this feature will be exposed in PL/Rust soon. However, it's still an extremely powerful feature for pure pgrx extensions as it allows calling another function directly, with runtime type safety checks, without incurring any Spi overhead.
cargo pgrx install --sudo
Sometimes cargo-pgrx install
can be tough to work with if targeting a package manager-managed Postgres installation as the various extension directories are owned by root. cargo pgrx install --sudo
fixes that by copying the files as root. The build, which is typically done as a regular user, is still run by that user -- only the file copying is run with sudo
.
cargo pgrx install --sudo
by @eeeebbbbrrrr in https://github.com/pgcentralfoundation/pgrx/pull/1322
Thanks to all the contributors! We appreciate the PRs and testing. And we especially like hearing about what you're building with pgrx.
Full Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.10.2...v0.11.0
Welcome to pgrx v0.10.2!
In pgrx v0.10.0, as part of a refactoring step in order to make refactoring easier in the future, without things like losing track of random functions, we... omitted a few functions! These depended on the cshim, but were not direct bindings, themselves. Oops. They're back now in pgrx::pg_sys
.
cargo pgrx
works now for Postgres 16 on macOSInstalling extensions on macOS for Postgres 16 had problems due to recent changes to Postgres defaults. We have fixed this and expanded our test coverage for macOS to reduce the odds of this happening in the future, thanks to @bayandin in https://github.com/pgcentralfoundation/pgrx/pull/1300
There were some problems with composite types and arrays, and specifically arrays of composite types in composite types. These are the sorts of things that cause edge cases in PgHeapTuple. They should now be appropriately supported, however, thanks to @eeeebbbbrrrr in https://github.com/pgcentralfoundation/pgrx/pull/1303
#[pg_schema]
will enable your bad ideasThe following should now work appropriately:
#[pgrx::pg_schema]
mod pg_catalog {
It does what you think it does. Since this can affect a global namespace in a way that makes it much easier to conflict with other extensions, it is best to avoid this. However, it is the sort of bad idea that extensions which "take over" the Postgres host to extensively redefine the host's capabilities, like ZomboDB does, may have to resort to in order to accomplish their goal. Thanks to @eeeebbbbrrrr in https://github.com/pgcentralfoundation/pgrx/pull/1302
And thank you for reporting your issues!
This is pgrx v0.10.1. It includes support for Postgres v16 and since pg16 is now released, cargo pgrx init
will now be able to download the latest version.
As always, please install the latest cargo-pgrx with cargo install cargo-pgrx --version 0.10.1 --locked
and update your Cargo.toml dependencies.
#1295: Support the official Postgres v16 release #1294: Add WAL recovery header bindings for pg15 and pg16
Full Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.10.0...v0.10.1
This is the final release of v0.10.0. Thanks everyone for the beta testing, pull requests, issues, and patience.
As always, install cargo-pgrx
with cargo install cargo-pgrx --locked
and update your extension Cargo.toml files to use the 0.10.0
pgrx dependencies.
This release includes support for Postgres 16RC1. Support for the previous betas has been removed. As such, a fresh cargo pgrx init
is required.
GetMemoryChunkContext
port by @workingjubilee in https://github.com/pgcentralfoundation/pgrx/pull/1273
pg_config
isn't found. by @eeeebbbbrrrr in https://github.com/pgcentralfoundation/pgrx/pull/1271
PostgresHash
also need Eq
by @workingjubilee in https://github.com/pgcentralfoundation/pgrx/pull/1264
Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.10.0-beta.4...v0.10.0
v0.10.0's full set of changes throughout the entire beta period are:
pg_sys::Datum
into a pgrx::Date
by @eeeebbbbrrrr in https://github.com/pgcentralfoundation/pgrx/pull/1177
AnyNumeric
is no longer backed by Postgres-allocated memory by @eeeebbbbrrrr in https://github.com/pgcentralfoundation/pgrx/pull/1216
#[pg_test]
-generated tests. by @thomcc in https://github.com/pgcentralfoundation/pgrx/pull/1239
Result<T, _>
by @eeeebbbbrrrr in https://github.com/pgcentralfoundation/pgrx/pull/1241
From
impls by @workingjubilee in https://github.com/pgcentralfoundation/pgrx/pull/1253
cppflags
it needs (on macOS, anyway) by @thomcc in https://github.com/pgcentralfoundation/pgrx/pull/1247
Full Changelog: https://github.com/pgcentralfoundation/pgrx/compare/v0.9.8...v0.10.0