Expose a lifecycle notification when an invocation pauses after exhausting retries

[LastRemote]

Problem

When an invocation retry policy is configured with:

InvocationRetryPolicy(
    max_attempts=10,
    on_max_attempts="pause",
)

Restate pauses the invocation after the final failed attempt, but there is no application-visible callback or pushed lifecycle notification for that transition.

The handler does not receive a final exception when the invocation is paused, so it cannot reliably emit an event such as:

INVOCATION_PAUSED

The paused state can be discovered afterward through introspection or the Admin API, but doing so requires polling.

Use case

We use Restate to run LLM-backed application workflows.

Transient failures inside ctx.run_typed are emitted by the run callback for observability. After ten failed invocation attempts, Restate pauses the invocation so that an operator can investigate and resume it.

At that point, the application needs to notify the client that:

• automatic retries have stopped;
• the invocation is paused;
• manual intervention or retry is available;
• the last failure was a particular provider or application error.

Currently, the application can observe the individual transient errors, but it cannot observe the authoritative transition from retrying to paused.

Current behavior

attempt 1 fails
...
attempt 10 fails
Restate transitions invocation to paused
no handler code executes
no documented push notification is emitted

The only reliable way for another application component to discover the transition is to repeatedly query invocation state.

Requested behavior

Expose the transition to paused as a durable lifecycle event containing at least:

• invocation ID;
• service and handler target;
• pause reason;
• whether the pause was caused by on_max_attempts;
• configured or consumed attempt count;
• last failure error code and message;
• related journal command or run name, when available.

Ideally, users should have a non-polling way to consume this event, such as:

• a configured Restate service handler;
• a lifecycle event or CDC stream;
• a webhook;
• an explicit OpenTelemetry event or span attribute intended for this purpose.

As a minimum, a Paused or PausedAfterMaxAttempts entry in sys_journal_events would make the transition durably inspectable and would align with the work being done for onMaxAttempts: "kill" in #4425 / #4560. However, a journal entry alone would not solve the non-polling notification use case.

Why handler-level workarounds are insufficient

An outer exception handler cannot observe transient ctx.run_typed failures because those retries are managed internally by Restate.

When invocation-level max_attempts is exhausted with on_max_attempts="pause", Restate does not raise a final exception into the handler. Therefore, application code cannot reliably run a final notification step at the pause boundary.

The application can infer that ten failures probably result in a pause, but Restate remains the authoritative owner of the invocation state. The application should not have to reproduce or infer that state transition independently.