Skip to content

Heads-up: release-mode crash when co-linked with swift-transformers (Tokenizers); upstream tracker swiftlang/swift#88794 #646

@aurascoper

Description

@aurascoper

Heads-up: release-mode crash when co-linked with huggingface/swift-transformers (Tokenizers)

Filing as a courtesy / FYI — this looks like a Swift release-mode codegen issue, not a defect in postgres-nio. Upstream tracker: swiftlang/swift#88794.

Symptom

A Swift release build that links both postgres-nio (1.33.0) and huggingface/swift-transformers (Tokenizers product, 1.3.0) into the same executable crashes with freed pointer was not the last allocation on the second consecutive PostgresClient.query(...) call. The first query succeeds; the second aborts the process.

Either dependency on its own works fine in release. A swift build -c debug build with both deps also works fine.

The trigger is import Tokenizers itself — no Tokenizers API has to be called for the crash to fire.

Environment

  • Swift 6.3.1 (swiftlang-6.3.1.1.2) / Xcode 26.4.1 / macOS 26.4.1, Apple M4
  • postgres-nio 1.33.0
  • swift-transformers 1.3.0
  • swift-nio 2.99.0

Minimal repro

The full minimal Package.swift + main.swift (≈30 lines) is in swiftlang/swift#88794. Two SELECT N queries are enough to reproduce.

Why I'm posting here

Pure FYI in case:

  1. There's a known interaction between NIO's allocator paths and Tokenizers' module-load code that the postgres-nio side has ideas about, or
  2. you want this on your radar in case other users hit it.

I'm not asking postgres-nio to fix this; the diagnosis points at the Swift toolchain. Feel free to close after triage. If a workaround on the postgres-nio side is feasible, that'd be a nice-to-have, but a Swift compiler patch is the right durable fix.

Workarounds for users hitting this today

  • Build the executable with swift build -c debug when both libraries are linked.
  • Avoid linking Tokenizers and PostgresNIO into the same executable target (split into two executables that talk over IPC, or keep tokenization in a separate process).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions