Nx TUI/PTTY intermittently swallows child-process failures during parallel run-many builds

[MmtBkn]

Summary

When running multiple projects in parallel with Nx’s default TUI-enabled output path, individual tasks can intermittently fail with no error output. Nx reports the affected task as failed or flaky, but the underlying signal or process failure is not surfaced.

The failing project is not consistent between runs.

The same command, with the same projects, parallelism, cache state, and available system resources, runs reliably when the TUI is disabled using:

nx run-many -t build --tui=false

or:

NX_TUI=false nx run-many -t build

This appears to be caused by the pseudo-terminal execution path swallowing or masking child-process termination information.

Environment

• Nx monorepo
• Multiple Next.js applications
• Approximately 11 nextjs projects executed through nx run-many -t build
• macOS
• Parallel task execution enabled
• Nx TUI enabled by default
• Reproduced on a clean, committed working tree with no concurrent file modifications
• NPM: 10.9.4

NX   Report complete - copy this into the issue template

Node           : 22.21.1
OS             : darwin-arm64
Native Target  : aarch64-macos
npm            : 10.9.4
daemon         : Available

nx                     : 22.5.4
@nx/js                 : 22.5.4
@nx/eslint             : 22.5.4
@nx/workspace          : 22.5.4
@nx/cypress            : 22.5.4
@nx/devkit             : 22.5.4
@nx/eslint-plugin      : 22.5.4
@nx/module-federation  : 22.5.4
@nx/next               : 22.5.4
@nx/playwright         : 22.5.4
@nx/react              : 22.5.4
@nx/rollup             : 22.5.4
@nx/storybook          : 22.5.4
@nx/vite               : 22.5.4
@nx/vitest             : 22.5.4
@nx/web                : 22.5.4
@nx/webpack            : 22.5.4
typescript             : 5.9.3
---------------------------------------
Registered Plugins:
xxx
---------------------------------------
Local workspace plugins:
xxx
---------------------------------------
Cache Usage: 2.34 GB / 92.64 GB

Steps to Reproduce

1. Create or use an Nx workspace containing several independently buildable projects.
2. Run all builds in parallel with the default TUI behavior:

nx run-many -t build

3. Repeat the command multiple times.
4. Observe that an arbitrary project may occasionally fail without any meaningful stderr or task output.
5. Run the same workload with the TUI disabled:

nx run-many -t build --tui=false

6. Repeat the non-TUI command multiple times.

Actual Behavior

With the TUI enabled:

• An arbitrary task intermittently fails.
• Nx may classify the task as flaky.
• The task produces no actionable error output.
• A different project may fail on subsequent runs.
• Available memory does not correlate with the failure.
• Increasing or limiting application-level worker counts does not eliminate the issue.

Controlled comparison on a stable working tree:

TUI disabled: 8 passes, 0 failures
TUI enabled: intermittent failure

The workload successfully completed with approximately 0.46 GB of free RAM when the TUI was disabled, but failed with approximately 3.94 GB free when the TUI was enabled. This indicates the failure is not explained by general memory pressure.

Expected Behavior

Nx should:

1. Preserve and report the actual child-process exit code or terminating signal.
2. Include the affected process’s stderr and available diagnostic output.
3. Produce equivalent task success or failure behavior regardless of whether TUI rendering is enabled.
4. Avoid converting a child-process or pseudo-terminal failure into a silent task failure.
5. Clean up pseudo-terminal resources and shutdown callbacks after each task completes.

Suspected Root Cause

The behavior appears related to Nx’s pseudo-terminal task execution path.

Based on inspection of Nx’s task-runner implementation:

• forkProcess selects the pseudo-terminal branch when TUI is enabled.
• The non-TUI exit path explicitly propagates certain child-process exit codes.
• The equivalent propagation does not appear to occur when TUI is enabled.
• The PTY process exits successfully when its pseudo-IPC channel closes, potentially masking the actual child-process termination.
• Nx allocates pseudo-terminal shutdown callbacks per task, but they do not appear to be pruned after task completion.

The combined behavior could explain why:

• The task is marked as failed.
• The failure has no associated output.
• The affected task changes between executions.
• Disabling the TUI avoids the issue without changing task parallelism.

Relevant implementation areas include:

packages/nx/src/tasks-runner/forked-process-task-runner.ts
packages/nx/src/tasks-runner/fork.ts

The exact paths may differ by Nx version.

Workaround

Disable the Nx TUI globally:

{
  "tui": {
    "enabled": false
  }
}

or per invocation:

nx run-many -t build --tui=false

The TUI can be explicitly re-enabled when needed using:

nx run-many -t build --tui

or:

NX_TUI=true nx run-many -t build

Acceptance Criteria

• Child processes terminated by a signal while using the TUI path report the original signal or a corresponding non-zero exit code.
• Nx displays actionable stderr or diagnostic output for the failed task.
• The PTY wrapper does not exit with code 0 when the underlying task failed.
• Parallel run-many behavior is functionally equivalent with TUI enabled and disabled.
• Pseudo-terminal resources, sockets, event handlers, and shutdown callbacks are removed after each task completes.
• Automated tests cover a child process that:
  • exits normally with a non-zero code;
  • is terminated by a signal;
  • closes its IPC channel unexpectedly;
  • runs as one of several parallel tasks.
• A regression test verifies that the task failure is surfaced when TUI is enabled.
• A regression test verifies that repeated parallel executions do not accumulate PTY shutdown callbacks or related resources.

Suggested Testing

Automated test

Add task-runner integration coverage that launches several parallel tasks, deliberately terminates one child process, and asserts that:

• the correct project is marked as failed;
• the terminating signal or exit code is retained;
• diagnostic output is displayed;
• the Nx parent process returns a non-zero exit code;
• unaffected parallel tasks are handled consistently;
• no PTY callbacks or sockets remain registered after completion.

Run the test repeatedly to expose race conditions:

for i in {1..50}; do
  nx run-many -t test-pty-task-runner --tui=true || exit 1
done

Manual verification

Run a representative parallel build repeatedly with both modes:

for i in {1..20}; do
  nx run-many -t build --tui=true || break
done

for i in {1..20}; do
  nx run-many -t build --tui=false || break
done

Confirm that failures are either eliminated or produce the same complete, actionable diagnostics in both modes.

Additional Context

This was originally investigated as possible memory pressure or excessive Next.js build concurrency. Those explanations were not supported by controlled testing.

Application worker limits may reduce resource usage, but they do not resolve the silent failure behavior. The only variable that consistently changed the outcome was whether Nx used its TUI/pseudo-terminal execution path.

Likely the same root cause as #31307