fix(pty): replace unsafe forkpty with posix_spawn on Unix

[0xStuart]

Pull Request Summary: Replace Unsafe forkpty with posix_spawn on Unix

This Pull Request resolves the immediate startup crashes and memory corruption issues encountered when spawning a local PTY process on Linux.

The Problem (Root Cause)

Originally, UnixPty.cs used the forkpty API from libutil (Linux) and libSystem (macOS). Calling a raw POSIX fork() in a multi-threaded .NET application is unsupported by the CLR.
When fork() clones the process:

1. Only the calling thread is duplicated in the child process.
2. All other threads (such as the GC helper threads, JIT compilation threads, and finalizer threads) do not exist in the child.
3. However, all internal mutexes, locks, and allocation states of the .NET runtime remain in their "locked" state in the child process memory.

Even when attempting to execute zero managed instructions in the child thread immediately post-fork, returning from the system clone call in the parent thread corrupts the CLR JIT compilation/page protection table state on Linux. This r

The Solution (posix_spawn)

To avoid unsafe process cloning inside the multi-threaded C# host entirely, we migrated Unix process spawning to the standard POSIX posix_spawn API.

1. Controlling Terminal (tty) Association

A major hurdle when using posix_spawn with PTYs is associating the spawned child process session with the PTY master/slave.

• By setting the POSIX_SPAWN_SETSID attribute, the child process is spawned as a new session leader.
• We configure a file action using posix_spawn_file_actions_addopen to open the slave PTY device path (e.g. /dev/pts/X) as fd 0 (stdin) inside the child process without the O_NOCTTY flag.
• According to UNIX terminal specifications, when a session leader opens a tty device without O_NOCTTY, the tty automatically becomes the controlling terminal of the session.
• This allows standard PTY line signals (like SIGINT / Ctrl+C and SIGTSTP / Ctrl+Z) to propagate correctly to foreground processes and job control to function natively.

2. Environment & Spawning Details

• Standard Descriptors: The slave PTY descriptor is duplicated to stdout (fd 1) and stderr (fd 2) in the child via posix_spawn_file_actions_adddup2.
• Working Directory: The child's startup working directory is safely configured by temporarily changing the parent thread's current directory context before the synchronous posix_spawn call and restoring it in a finally block im
• Direct Shell Launch: We launch the shell executable directly, completely removing the process-startup latency associated with intermediary /bin/sh -c wrappers.

Deleted Boilerplate & Simplifications

By moving the spawn orchestration to the OS kernel level before executing the target binary, we safely removed several complex, error-prone C# workarounds:

1. LoadLibcHandle: Replaced by standard [DllImport("libc")] which is resolved automatically on both macOS and Linux by the .NET runtime.
2. GetSignalsToResetForExec: Manual signal disposition resets are no longer needed because the OS kernel automatically handles resetting caught signal handlers to defaults upon process execution.
3. AllocNativeString / FreeNative: Since memory allocation and preparation occur synchronously on the parent thread before execution, standard unmanaged allocations are safely released using standard C# finally blocks rather t

Multi-Platform Safety

• Windows: Completely unaffected (uses native Windows ConPTY API implementation located in RoyalTerminal.Terminal.Pty.Windows).
• macOS and Linux: Share the updated UnixPty.cs. Dynamic platform checks align compile-time constants (e.g. POSIX_SPAWN_SETSID and O_NOCTTY values) to ensure correctness on both Darwin and Linux kernels.

Verification & Tests

All 702 unit and integration tests run successfully, including:

• Interactive shell startup and prompt synchronization.
• PTY terminal resizing, output flow control, and terminal mode changes.
• Headless Avalonia FACT interaction tests verifying Ctrl+C and Ctrl+Z signal latency budgets under massive flood cycles.

[StefanKoell]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8823d57f1c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Avoid mutating the process-wide working directory

When two terminal sessions start concurrently with different workingDirectory values, this process-global mutation can interleave: one child can spawn in the other session's directory and the final restore can leave the entire application in the wrong directory. Other threads can also resolve relative paths against the temporary directory during this window. Set the child directory through a spawn/file action (or equivalent child-only mechanism) rather than changing Directory.CurrentDirectory in the host.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Restore terminal signal defaults in the spawned child

When the host inherits ignored terminal-control signals (for example, SIGINT or SIGTSTP), exec preserves those ignored dispositions. The removed fork path explicitly reset them, but this spawn path only sets POSIX_SPAWN_SETSID; as a result Ctrl+C/Ctrl+Z can be ignored by interactive shells in that environment. Configure POSIX_SPAWN_SETSIGDEF with the terminal-control signals (and check the attribute call result) before spawning.

Useful? React with 👍 / 👎.