A synchronous step failure never reaches the executions error surface

[nathanpond]

Discovered while working #168.

What

WorkflowExecutionErrorRecorder (src/AutoNate.Web/Services/Workflow/WorkflowExecutionErrorRecorder.cs:19) records a workflow_execution_errors row for exactly one event type:

private const string FailedEventType = "job.execution.failed";

That is the only thing feeding the executions error surface — the red node on the diagram, isErrored / errorMessage / errorCount in GET /api/executions/{id}/history.

A step that fails synchronously never produces a job, so it can never produce a job.execution.failed, so it can never appear on that surface. The failure takes down the API call that triggered it (a 500 on start, or on completing the task in front of it) and leaves no trace in the execution's own history.

How it surfaced

#168's verify-first probe against Flowable 8.0.0, comparing a failing service task with and without flowable:async:

• unmarked — POST /runtime/process-instances returns 500, and GET /history/historic-process-instances shows no instance at all. Nothing to attach an error to.
• marked — the step becomes a job, is retried, and lands in /management/deadletter-jobs with retries=0 and its exception intact.

So today, marking a step as a retry point is also what makes its failure visible. That is a surprising coupling: an author choosing a checkpoint for retry semantics is unknowingly also choosing observability.

Also observed, and worth separating

In the E2E environment the async failure dead-lettered reliably within seconds, but no workflow_execution_errors row appeared within 45 seconds. That may be nothing more than the Dapr/NATS hop not being wired for the app instance the E2E suite drives — unlike the design gap above, it is not established. Worth confirming before treating it as a second defect.

Why not fixed in #168

#168 is scoped to the retry-point setting, and its AC asks only that the job is produced and that the existing error path is not regressed — neither of which this touches. The operator-facing job surface is M5's story, and this finding is most useful as input to it: the question "which failures can an operator see?" currently has the answer "only the ones that happened to be asynchronous."

[nathanpond]

Triage: not an M4 story — input to M5's operator-visibility work

Proposal from re-planning, stated so it can be overridden.

This is the reason #168 carries an unticked criterion ("…and the execution's error surface reports it"). The dead-lettering half is asserted; the error-surface half is what this issue says does not happen.

Why not fix it in M4: the fix is a change to what feeds the executions error surface, which is the subject of M5's operator-visibility story rather than of any element-coverage story here. Doing it under an M4 story would mean an element story quietly rewriting the failure-reporting pipeline.

Why it should not simply wait: the coupling it describes is surprising — as things stand, marking a step as a retry point is also what makes its failure visible, because only job.execution.failed feeds that surface and a synchronous failure never produces a job. An author choosing a checkpoint for retry semantics is unknowingly also choosing observability.

Left open and labelled for M5. #168's criterion stays unticked until then, which is the honest state.