Skip to content

fix(session-sqlx): let failed executions re-run - #1531

Closed
0xAlcibiades wants to merge 1 commit into
paradigmxyz:mainfrom
0xAlcibiades:alcibiades/let-failed-executions-re-run
Closed

fix(session-sqlx): let failed executions re-run#1531
0xAlcibiades wants to merge 1 commit into
paradigmxyz:mainfrom
0xAlcibiades:alcibiades/let-failed-executions-re-run

Conversation

@0xAlcibiades

Copy link
Copy Markdown
Contributor

Closes #1507

Context

The unique idempotency index on session_executions covered every row, so a
failed or cancelled execution permanently held its (thread_key, idempotency_key) pair. For any trigger with a stable key the work then became
un-rerunnable: the insert hit the index and the caller got the dead row back
with created = false.

A GitHub review is keyed per commit, so this is reachable in normal operation.
A turn killed mid-flight — a control-plane roll, say — leaves a failed row
that no identical re-request can ever get past. The only recovery is editing the
row by hand or pushing a new commit to move the key.

Change

Migration 0053 swaps the index in place so it covers only the statuses where
dedupe is still meaningful:

create unique index if not exists session_executions_active_idempotency_idx
    on session_executions (thread_key, idempotency_key)
    where idempotency_key is not null
      and status in ('queued', 'running', 'completed');

In-flight and completed work still dedupes, so a retried webhook cannot start a
second live turn or resurrect a finished answer. Failed and cancelled rows drop
out of the index, and the next insert with that key creates a fresh execution.
The upsert conflict clause is narrowed to match the new index.

Testing

  • cargo fmt --all --check and cargo clippy -p centaur-session-sqlx --all-targets: clean.
  • cargo test -p centaur-session-sqlx: passes.

One caveat worth stating plainly: the new test,
failed_execution_does_not_block_rerun_on_same_idempotency_key, is
database-backed and returns early when SESSION_RUNTIME_TEST_DATABASE_URL is
unset. I had no Postgres available, so it reported ok without executing, and
I have not exercised the migration or the new index locally.
CI has the
database and will actually run it. The test asserts that a failed execution is
followed by a fresh created = true execution on the same key.

0053 is the next free number on top of 0052_session_execution_request.sql at
the time of writing; it will need renumbering if another migration lands first.

The unique idempotency index on session_executions covered every row, so a
failed or cancelled execution permanently blocked a fresh execution that
carried the same (thread_key, idempotency_key). For triggers with a stable
key per commit, a turn killed by a control-plane roll became un-rerunnable:
the insert hit the index and the caller received the dead row with
created = false.

The index now covers only queued, running, and completed rows. In-flight and
completed work still dedupes; failed and cancelled work falls out of the
index, so the next insert creates a fresh execution. Migration 0053 swaps the
index in place and the upsert conflict clause is narrowed to match it.
@mslipper

mslipper commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Thanks for looking into this. I don’t think we should merge this. It weakens the session API’s global idempotency contract: failed executions may already have produced external side effects, so allowing the same request key to create another execution can duplicate work.

I opened #1603 with a GitHub-specific fix instead. It keys explicit review requests by GitHub delivery ID, so a fresh request can run again while redeliveries remain idempotent. I’m going to close this PR in favor of that one.

@mslipper mslipper closed this Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

failed executions permanently hold their idempotency key; identical re-requests return the dead execution

3 participants