Releases: pgcentralfoundation/pgrx
v0.12.0-beta.0
Hello!
NOTE: You should actually upgrade to
In pgrx 0.12.0-beta.0 0.12.0-beta.5, the first news of import is that if you run
cargo install cargo-pgrx --version 0.12.0-beta.5 --locked
You should then be able to run
cargo pgrx upgrade --to 0.12.0-beta.5
in your project to update all of the versions of pgrx libraries in your project's Cargo.tomls. Please let us know how it goes!
Many cargo-pgrx updates and modifications
- only build library in cargo-pgrx commands by @usamoi in #1643
- Fix cargo pgrx finding cdylibs by @workingjubilee in #1636
- Cargo pgrx upgrade by @NotGyro in #1687
cargo pgrx test --runas
by @eeeebbbbrrrr in #1652- Use
CARGO_TARGET_DIR
if available by @workingjubilee in #1662 - Only suggest
cargo install
with 1 version by @workingjubilee in #1659 - Use supports-colors feature by @theory in #1663
- Allow relocatable extensions by @NotGyro in #1653
Internal improvements
- @usamoi made most of our generated bindings
#[inline]
in #1616 which will defer codegen until the FFI guard is actually used, which should improve build times considerably - Memory management for Datum to String conversion by @sardination in #1684 should reduce allocations
- Use Ceescape's
sigsetjmp()
wrapper by @NotGyro in #1621 takes care of a UB concern wheresigsetjmp
andsiglongjmp
aren't concepts that can be reflected in Rust code, and thus must be called when Rust "isn't looking": insideasm!
. Without this, these calls may be miscompiled, because LLVM may strip the "returns-twice" codegen annotation from them. - @JelteF fixed an odd case where
OnceLock
improperly modeled the lifetime of a Postgres extension, which may persist between resets of the server by using UnsafeCell for PgLwLock and PgAtomic in #1696
New API
- Nullable type by @NotGyro in #1602
- Add one-or-none iterator conveniences by @workingjubilee in #1703
- If you enable pgrx with
features = ["nightly"]
, there is now animpl Allocator for MemCx
from @NotGyro in #1737
Time breakage
Several of our datetime APIs were incorrect in various ways. On the positive side, the remaining functions should no longer panic.
- Break Time on the anvil of proptest by @workingjubilee in #1610
- There are 1000 microseconds per millisecond by @workingjubilee in #1639
New FFI bindings
We have added many more bindings to the cshim and pgrx-pg-sys!
- feat: add more cshim bindings by @rebasedming in #1628
- feat: add
ExecCopySlotHeapTuple
topg_sys
by @rebasedming in #1635 - Redeclare additional *_walker fn by @daamien in #1629
- feat: add bindings for WAL-related functions by @rebasedming in #1675
- add user_mapping.h by @rebasedming in #1742
- add bindings of fmgroids.h, OIDs of built-in functions by @usamoi in #1726
Breakage of many functions
The pgrx::srf
module, which contained no visible code, is no longer accessible.
A number of changes in a hidden module have been done, in conjunction with alterations to the supporting code that #[pg_extern]
generates, to make the process of "unboxing" arguments and "boxing" returns... transforming them to and from Datums, specifically at function boundaries... sound. If your code has been broken, it either is a rare and specific SetOfIterator or TableIterator type or it was probably unsound, especially if it used the 'static
lifetime. Please report any unexpected breakages.
Fortunately, this has upsides beyond just soundness: now lifetimes in pgrx functions should simply work, and work correctly.
The PRs of interest:
- Implement
RetAbi
andBoxRet
by @workingjubilee in #1701 - Allow real borrow-checking of
#[pg_extern] fn
lifetimes by @workingjubilee in #1724 - Fix lifetimes for IO functions by @workingjubilee in #1733
- FcInfo wrapper by @NotGyro in #1720
- Make unboxing args sound with
ArgAbi
by @workingjubilee in #1731
Documentation improvements
If you use mdbook you can generate our documentation. Unfortunately it isn't published yet. We've been making various small improvements to it.
- inline lib reqs for
cargo pgrx init
by @jyn514 in #1630 - Document what happens with the
./sql
directory by @eeeebbbbrrrr in #1725 - document pg_guard to the best of my abilities by @eeeebbbbrrrr in #1729
Span adjustments for diagnostics/linting
A number of fixes have been done to how pgrx generates code so that the code should now be detectable as macro-expanded code while still pointing to the appropriate code in errors. There are still diagnostic improvements to be had and this is not done uniformly across all of pgrx yet, so please report if there are issues with these.
- Pipe forward spans by formatting idents by @workingjubilee in #1665
- Ad hoc span fixups by @workingjubilee in #1668
- Begin synthesizing spans by @workingjubilee in #1746
Deprecations
A number of things have been deprecated gracefully rather than simply unceremoniously dumped.
- Deprecate
set_varsize
by @workingjubilee in #1617 - Deprecate
variadic!
support by @workingjubilee in #1713 - C enums as module consts + deprecation by @workingjubilee in #1748
New Contributors
- @jyn514 made their first contribution in #1630
- @my-vegetable-has-exploded made their first contribution in #1655
- @staticle made their first contribution in #1656
- @theory made their first contribution in #1663
- @sardination made their first contribution in #1684
- @hamirmahal made their first contribution in #1717
Full Changelog: v0.12.0-alpha.1...v0.12.0-beta.0
v0.11.4
Welcome to pgrx v0.11.4!
This is a minor release to add some additional functionality to cargo-pgrx
to enable running tests in atypical CI configurations and the ability to turn off its colored output via the NO_COLOR
environment variable. This minor release also includes a small change that should measurably improve compilation times for all extensions.
To update to this this, please run cargo install cargo-pgrx --version 0.11.4 --locked
and ensure to update your extension's Cargo.toml
file to use pgrx v0.11.4.
What Changed
cargo-pgrx
changes
PR #1652 was cherry-picked into this release to include the new --runas
and --pgdata
arguments to cargo pgrx test
. --runas
uses the system sudo
command to run the temporary Postgres cluster process as the specified user, and uses --pgdata
as the location where the cluster's PGDATA directory should be initialized.
cargo-pgrx
now understands the NO_COLOR
and FORCE_COLOR
environment variables supported by the upstream dependency it uses for generated colored output. See https://crates.io/crates/owo-colors for details. This is from PR #1664
Compilation Times
pgrx generates a (very large) set of Rust bindings that represent the "rust version" of all the internal Postgres symbols (functions, globals, etc). In the case of functions, pgrx also generates wrapper functions so it can instrument calls with the proper guard rails. It turns out, simply annotating these wrapper functions with #[inline]
drastically improves compilation time as it allows the rust compiler to defer their compilation until the very end. Which ultimately means only the functions used by an extension get compiled -- the others are DCE'd. This came from PR #1616.
Thanks!
Thanks to @usamoi for improving our compilation times, and shout out to @theory for his first contribution!
Full Changelog: v0.11.3...v0.11.4
v0.12.0-alpha.1
Welcome to pgrx 0.12.0-alpha.1!
Say the magic words with me!
cargo install cargo-pgrx --locked --version 0.12.0-alpha.1
Breaking Changes
No more dlopen!
Perhaps the most exciting change this round is @usamoi's contribution in #1468 which means that we no longer perform a dlopen
in order to generate the schema. The cost, such as it is, is that your pgrx extensions now require a src/bin/pgrx_embed.rs
, which will be used to generate the schema. This has much less cross-platform issues and will enable supporting things like cargo binstall
down the line.
It may be a bit touchy on first-time setup for transitioning older repos. If necessary, you may have to directly add a src/bin/pgrx_embed.rs
and add the following code (which should be the only code in the file, though you can add comments if you like?):
::pgrx::pgrx_embed!();
Your Cargo.toml will also want to update its crate-type key for the library:
[lib]
crate-type = ["cdylib", "lib"]
Library Code
- pgrx-pg-sys will now use
ManuallyDropUnion
thanks to @NotGyro in #1547 - Due to other uses of that name,
ErrorReport::report
is nowunwrap_or_report
! #1568 - VARHDRSZ
const
s are no longerfn
, thanks to @workingjubilee in #1584 - We no longer have
Interval::is_finite
since #1594 - We translate more
*_tree_walker
functions to the same signature their*_impl
version in Postgres 16 has: #1596 - Thanks to @eeeebbbbrrrr in #1591 we no longer have the
pg_sql_graph_magic!()
macro, which should help with more things in the future!
What's New
We have quite a lot of useful additions to our API:
SpiClient::prepare_mut
was added thanks to @XeniaLu in #1275- @usamoi also contributed bindings subscripting code in #1562
- For
#[pg_test]
, you have been able to use#[should_panic(expected = "string")]
to anticipate a panic that contains that string in that test. For various reasons,#[pg_test(error = "string")]
is much the same. Now, you can also use#[pg_test(expected = "string")]
, in the hopes that is easier to stumble across, as of #1570
Result<composite_type!("..."), E>
support
- In #1560 @NotGyro contributed support for using
Result<composite_type!("Name"), E>
, as a case that had not been handled before.
Significantly expanded docs
Thanks to @rjuju, @NotGyro, and @workingjubilee, we now have significantly expanded docs for cargo-pgrx and pgrx in general. Some of these are in the API docs on https://docs.rs or the READMEs, but there's also a guide, now! It's not currently published, but is available as an mdbook in the repo.
Some diagnostic information that is also arguably documentation, like comments and the suggestion to cargo install
, have also been improved, thanks to @workingjubilee in
#[pg_cast]
An experimental macro for a CREATE CAST
was contributed by @xwkuang5 in #1445!
Legal Stuff
Thanks to @the-kenny in #1490 and @workingjubilee in #1504, it was brought to our attention that some dependencies had unusual legal requirements. So we fixed this with CI! We now check our code included into pgrx-using binaries is MIT/Apache 2.0 licensed, as is common across crates.io, using cargo deny
!. The build tools will have more flexible legal requirements (partly due to the use of Mozilla Public License code in rustls).
Internal Changes
Many internal cleanups were done thanks to
- @workingjubilee in too many PRs to count!
- @thomcc found a needless condition in #1501
- @nyurik in too many PRs to count!
In particular:
- we now actually
pfree
ourArray
s we detoasted as-of #1571 - creating a
RawArray
is now low-overhead due to #1587
Soundness Fixes
We had a number of soundness issues uncovered or have added more tests to catch them.
- Bounds-checking debug assertions for array access by @NotGyro in #1514
- Fix unsound
&
and&mut
infcinfo.rs
by @workingjubilee in #1595
Less Deps
Part of the cleanup by @workingjubilee was reducing the number of deps we compile:
Hopefully it will reduce compile time and disk usage!
New Contributors
- @the-kenny made their first contribution in #1490
- @xwkuang5 made their first contribution in #1445
- @rjuju made their first contribution in #1516
- @nyurik made their first contribution in #1533
- @NotGyro made their first contribution in #1514
- @XeniaLu made their first contribution in #1275
Full Changelog: v0.12.0-alpha.0...v0.12.0-alpha.1
v0.12.0-alpha.0
Hello and welcome to The Beginning of The End!
You may have seen some largely-inscrutable rambling about "object lifetimes" and "memory contexts" from me in various comments, logs, and issues. You may have also heard that this has effectively stalled all forward feature work. You may have noticed this has been the case for months now.
The long and short of it is that pgrx never actually fully built-in the actual lifetimes of objects in Postgres into its various types. As more and more interlocking parts grew, between things like the type unboxing in trait FromDatum
, or the bounded memory context of Spi::connect
, eventually many supposedly-safe interfaces arose which proved wildly unsafe in reality due to this mesh of interactions. Internal portions of pgrx relied on expeditious hacks to ignore the growing problems, until this rotten foundation yielded an enormous crop of soundness issues over the year of 2023. Even when not exploitable, they prevent otherwise-trivial enhancements in terms of speed or memory usage.
Key elements of a new foundation have landed in pgrx, which allow ascribing lifetimes to every datum and every memory context that a Postgres extension will encounter. There is much more work to be done to actually make it so all Rust code has to flow through these new interfaces. This alpha release is an extremely "preview" release to allow assessing some of the damage from the first set of breaking changes, so that some initial feedback and triage can occur. There will, unfortunately, be more breaking changes on the horizon.
There are so many small changes that I am simply going to summarize the main point of them:
Many Macros Expand Subtly Differently
If your extension now breaks in some inscrutable way that you cannot otherwise discern, the problem is probably hidden in macro expansion. This cannot really be diagnosed except using cargo expand
diffed against previous expansions. A most likely issue is that something that previously hid a lifetime during expansion or writes it as 'static now propagates it by name or as the anonymous '_ lifetime.
Datum<'src>
There are now lifetime-bound Datums. Expect to see them in more places.
UnboxDatum<As<'src> = Type<'src>>
Expect to see this or a similar signature a lot: there are new conversion traits which use a generic associated type or similar sneaky tactics to allow "clamping down" the lifetime to a bound, unless the produced type is always 'static by being a simple value like i32
.
&'mcx MemCx<'_>
You will see more and more references to this type, which is a memory context that encodes an inner invariant lifetime, in signatures. If one of these is currently in-scope, then please remember it is always unsafe
to manipulate the static mut CurrentMemoryContext
in Postgres, but it is doubly so then. In general, when designing your own types using pgrx, you should prefer designing code that uses a signature like that of MemoryContextAlloc
instead of palloc
if you are at all uncertain.
Full Changelog: v0.11.0...v0.12.0-alpha.0
v0.11.3
The pgrx 0.11.3 release addresses several UB risks of pgrx, updates its dependencies to address possible security risks on a few points, and includes considerable numbers of additional headers in the release. It should also now be easier to actually use cargo-pgrx on a mildly more interesting network configuration, though you may need to specify --no-default-features
with either --features native-tls
or --features rustls
when you cargo install cargo-pgrx
.
New Bindings!
New bindings added thanks to
- @burmecia in #1432
- @daamien in
- @rebasedming in #1486
- @usamoi in #1436
- @workingjubilee in #1453
"...wait, that's UB?"
Two UB fixes!
Ergonomics
- A better
ereport!
macro in #1472
Less transport-level security problems in cargo-pgrx
- We no longer secretly require rustls thanks to @jirutka in #1448
- We now use native certs if possible, even with rustls, in #1449
Together these should mean it's possible to actually use cargo-pgrx on whatever your network configuration is, but you might have to use cargo install --no-default-features --features native-tls
to install with native-tls (which, on Linux, means OpenSSL). By default, you will use rustls.
Many tiny dependency updates
These address some largely-hypothetical security risks, but one is particularly important: the bindgen update means we now should be compatible with some aarch64 builds that might have failed.
Full Changelog: v0.11.2...v0.11.3
v0.11.2
pgrx v0.11.2 is a minor release which
- makes available the storage-related API, thanks to @silver-ymz in #1409
- deprecates the
Oid::from_u32_unchecked
API... because it is actually possible to do via casting in a query, which makes it effectively impossible for us to not wind up providing it via some other safe API, since the source is also an easily-Copied type. Thus you can now simply useFrom::from
for it. Thanks to @thomcc for this discovery, implemented by @workingjubilee in #1374
As usual, cargo install cargo-pgrx --version 0.11.2 --locked
and be on your merry way!
Thanks everyone!
Full Changelog: v0.11.1...v0.11.2
v0.11.1
Hello. Welcome to pgrx v0.11.1, a bugfix release that hopefully addresses a very annoying persistent problem for users, especially ones that were trying to use pgrx with Postgres 16! My (@workingjubilee)'s changes in b992f55 now steer bindgen towards including the exact compiler-header directory that we require, instead of relying on clang-sys to guess the right directory in ways that can go horribly wrong. Unfortunately, this may make our build step more brittle in certain cases. Please report issues as they arise, so that we can continue to refine our build system, and eventually fix this in bindgen proper!
This new behavior can still be disabled by setting PGRX_BINDGEN_NO_DETECT_INCLUDES
, as it is still a form of autodetecting includes, but if this release fixes your build that was previously relying on that or any other easily-removed hack, please let us know!
Also, thanks to @LuminousMonkey, you can now use pgrx on illumos, an operating system descended from Solaris, with no changes to your source code!
Full Changelog: v0.11.0...v0.11.1
v0.11.0
Welcome to pgrx v0.11.0. It's a new "major" release with some new features, a faster "build.rs", and some bugfixes.
As always, please update by running cargo install cargo-pgrx --version 0.11.0 --locked
and update your extension crate Cargo.toml
files accordingly.
What's Changed
Faster Buildings
@workingjubilee did some work to improve build times. As measured on their computer against cargo check
, it's roughly a quarter faster. In doing so, the generated "bindings.rs" files look a little different and our "build.rs" has been cleaned up a bit.
- Make check builds ~25% faster by merging extern blocks by @workingjubilee in #1316
- Touch up Display for PgNode by @workingjubilee in #1313
- Consistently order generated impls by @workingjubilee in #1314
- Organize bindgen arg sets in fn (nfc) by @workingjubilee in #1312
- Check for
is_for_release
once by @workingjubilee in #1315 - Report build versions that conflict by @workingjubilee in #1310
Pure Rust implementation for PgList
@workingjubilee did a tremendous amount of work creating a proper wrapper around Postgres' internal List
type, called PgList
.
- Introduce pure-Rust replacement for PgList by @workingjubilee in #1277
- Remove incorrect List::drain assert by @workingjubilee in #1283
- when constructing a new ListCell, the 'next' node begins life as NULL by @eeeebbbbrrrr in #1285
- Fix some List sizing bugs by @workingjubilee in #1289
- Actually use repalloc return in List by @workingjubilee in #1291
- Rename node_list.rs -> linked_list.rs by @workingjubilee in #1286
Some background here is that we intend to add the "dynamic function call" work (see below) to PL/Rust. In doing that work, use of Postgres' List
type is pretty important and pgrx' existing implementation was a) mostly in the cshim, and b) just not very-good-and-sound. Considering (b) and also that PL/Rust isn't able to use anything from pgrx' chsim (a), @workingjubilee set out to make a proper Rust wrapper/implementation of Postgres' List
type.
Note: As part of this, NodeTag
is now a proper enum
, which requires a small migration which is mostly satisfied by a regex-based find-replace of NodeTag_T_[A-Za-z]*
with NodeTag::T_$1
.
More Bindings
We've added more Postgres internals headers to the pgrx-pg-sys
crate:
- Generate bindings for WAL recovery functions by @feikesteenbergen in #1294
- Include bindings for controlfile by @feikesteenbergen in #1320
- Bind catalog header definitions for access method tables by @workingjubilee in #1306
Dynamic Function Calling
pgrx now allows calling any SQL-defined function (ie, via CREATE FUNCTION
) in a (mostly) Rusty way with the new dynamic function calling support. Some documentation around this feature can be found in the sources, here and in the unit tests.
- Dynamic function call by @eeeebbbbrrrr in #1279
As mentioned above, this feature will be exposed in PL/Rust soon. However, it's still an extremely powerful feature for pure pgrx extensions as it allows calling another function directly, with runtime type safety checks, without incurring any Spi overhead.
cargo pgrx install --sudo
Sometimes cargo-pgrx install
can be tough to work with if targeting a package manager-managed Postgres installation as the various extension directories are owned by root. cargo pgrx install --sudo
fixes that by copying the files as root. The build, which is typically done as a regular user, is still run by that user -- only the file copying is run with sudo
.
- Add
cargo pgrx install --sudo
by @eeeebbbbrrrr in #1322
Miscellaneous Work
- Replace cshimmed fn by @workingjubilee in #1304
- Enable proptest in more workflows by @workingjubilee in #1309
- Float Dockerfile's Fedora version by @workingjubilee in #1311
- Fix pg16 CI by @eeeebbbbrrrr in #1326
Thanks!
Thanks to all the contributors! We appreciate the PRs and testing. And we especially like hearing about what you're building with pgrx.
Full Changelog: v0.10.2...v0.11.0
v0.10.2
Welcome to pgrx v0.10.2!
In pgrx v0.10.0, as part of a refactoring step in order to make refactoring easier in the future, without things like losing track of random functions, we... omitted a few functions! These depended on the cshim, but were not direct bindings, themselves. Oops. They're back now in pgrx::pg_sys
.
cargo pgrx
works now for Postgres 16 on macOS
Installing extensions on macOS for Postgres 16 had problems due to recent changes to Postgres defaults. We have fixed this and expanded our test coverage for macOS to reduce the odds of this happening in the future, thanks to @bayandin in #1300
composites compose better
There were some problems with composite types and arrays, and specifically arrays of composite types in composite types. These are the sorts of things that cause edge cases in PgHeapTuple. They should now be appropriately supported, however, thanks to @eeeebbbbrrrr in #1303
#[pg_schema]
will enable your bad ideas
The following should now work appropriately:
#[pgrx::pg_schema]
mod pg_catalog {
It does what you think it does. Since this can affect a global namespace in a way that makes it much easier to conflict with other extensions, it is best to avoid this. However, it is the sort of bad idea that extensions which "take over" the Postgres host to extensively redefine the host's capabilities, like ZomboDB does, may have to resort to in order to accomplish their goal. Thanks to @eeeebbbbrrrr in #1302
Keep the bug reports coming
And thank you for reporting your issues!
v0.10.1
This is pgrx v0.10.1. It includes support for Postgres v16 and since pg16 is now released, cargo pgrx init
will now be able to download the latest version.
As always, please install the latest cargo-pgrx with cargo install cargo-pgrx --version 0.10.1 --locked
and update your Cargo.toml dependencies.
What's Changed
#1295: Support the official Postgres v16 release
#1294: Add WAL recovery header bindings for pg15 and pg16
Full Changelog: v0.10.0...v0.10.1