error handling

[tetron]

Want a way to specify that step failure isn't fatal for the workflow.

Proposal

steps:
  step1:
    onError: fail

Possible values of onError

• fail - existing behavior
• nullOut - output parameters are all 'null'
• retry - run step again

Discussion points:

• do we want a way to indicate error condition separately from producing 'null' ?
• if we include 'retry' should it (a) include a way to decide whether to retry or not (b) should there be a way to limit the number of retries (maybe this should be done with loops instead?)

Related:

Make successCodes, temporaryFailCodes and permanentFailCodes accept ranges in the form of "1-10"

[tetron]

Discussion:

An option to recover from errors should include a standard reporting mechanism.

Proposal:

onError behavior can be set at the workflow level or the individual step level.

If onError is true, when a step fails, produces "null" on all output parameters and information about the failed step is added to a logical "errors" list which is internal to the workflow runner.

If any errors ocurred, the output object includes a field "cwl:errors" which is an array objects providing basic information about each workflow step that failed.

{
  cwl:errors: [
    {
      stepId: URI for the step definition
      errors: [if this was a subworkflow, errors within the subworkflow]
      index: (scatter index or loop iteration counter, or null)
      jobId: platform-specific identifier for the specific job (provenance id, job uuid, etc)
    }
  ]
}

For cross-product scatters, might want to do 'index' differently.