Optimize PTY IO throughput and NativeAOT support

[wieslawsoltes]

Summary

This PR implements a Ghostty-style Unix PTY IO gather pipeline for RoyalTerminal, adds throughput fixtures and benchmark coverage, and makes the terminal/PTY stack publishable under NativeAOT.

The Unix PTY read path now splits kernel draining from subscriber dispatch:

• PTY-Gather owns nonblocking read(2)/poll(2) and fills a fixed ring of preallocated 64 KiB buffers.
• PTY-Dispatcher publishes completed batches through the existing DataReceived event.
• Short reads below 1024 bytes dispatch immediately to preserve interactive latency.
• Saturated 1024-byte reads bridge writer refill gaps with bounded spin/poll work before publishing.
• The public IPty contract stays unchanged.

Reference Review

The implementation was checked against the required terminal references before changing behavior:

• Ghostty src/termio/Exec.zig
• Windows Terminal ConptyConnection.cpp
• xterm.js WriteBuffer.ts

RoyalTerminal follows Ghostty for Unix PTY reads because the old implementation had the same serial read -> process/dispatch shape described by Ghostty. Windows ConPTY behavior is left unchanged because it has a different pipe/overlapped IO model and needs separate measurement.

What Changed

• Added a two-stage Unix PTY read pipeline with preallocated buffer reuse and backpressure.
• Added UnixPtyReadBatchPolicy and focused tests for batching thresholds and budget behavior.
• Added a large-output PTY contract test that verifies saturated output is delivered in batches larger than 1024 bytes.
• Added IO benchmark fixture generation for ASCII, mixed Unicode, and CSI-heavy payloads.
• Added --io, --generate-io-fixtures, --fixtures, --fixture-size-mb, --io-mode, and --io-repeats benchmark options.
• Added managed-VT PTY IO benchmarks that feed each batch through BasicVtProcessor and capture child cat wall time through /usr/bin/time -p.
• Added --vt-parse parser-only benchmarks plus callback timing/allocation columns for PTY IO rows.
• Added --ghostty, --ghostty-app, and --ghostty-path benchmark options to compare RoyalTerminal against an installed Ghostty build using the same fixture files.
• Recycled trimmed scrollback rows during steady-state whole-screen scrolling.
• Added printable ASCII fast paths for grapheme-append checks and codepoint width calculation.
• Added detailed implementation notes and validation log in docs/specs/terminal-io-throughput-optimization.md.

NativeAOT Support

• Enabled AOT compatibility analyzers for the terminal and PTY packages.
• Replaced reflection-based System.Text.Json serialization with source-generated metadata for terminal capture, command history, profiles, workspaces, and SSH secret payloads.
• Added a RoyalTerminal.PtyIoAotSmoke executable that publishes and runs through NativeAOT.
• Made native library probing AOT-safe by using AppContext.BaseDirectory instead of Assembly.Location.
• Added a demo NativeAOT publish path:
  • disables the optional Pretext pipeline under PublishAot=true;
  • excludes ReactiveUI.Avalonia under PublishAot=true;
  • replaces ReactiveWindow<T> activation with explicit Avalonia open/close lifetime disposal;
  • replaces trim-unsafe WhenAnyValue uses in the app controller with explicit property-change observables;
  • keeps off-thread settings-panel completion marshaled through Dispatcher.UIThread.

Validation

Validated locally on macOS arm64:

dotnet build samples/RoyalTerminal.Demo/RoyalTerminal.Demo.csproj -c Release
dotnet test tests/RoyalTerminal.Tests/RoyalTerminal.Tests.csproj -c Release
dotnet publish tests/RoyalTerminal.PtyIoAotSmoke/RoyalTerminal.PtyIoAotSmoke.csproj -c Release -r osx-arm64 -p:PublishAot=true --self-contained true -o /tmp/royalterminal-pty-aot-smoke-clean
/tmp/royalterminal-pty-aot-smoke-clean/RoyalTerminal.PtyIoAotSmoke
dotnet publish samples/RoyalTerminal.Demo/RoyalTerminal.Demo.csproj -c Release -r osx-arm64 -p:PublishAot=true --self-contained true -o /tmp/royalterminal-demo-aot
dotnet run --project tests/RoyalTerminal.Benchmarks/RoyalTerminal.Benchmarks.csproj -c Release -- --skip-render --io --fixture-size-mb 1 --fixtures /tmp/royalterminal-io-fixtures-smoke --output /tmp/royalterminal-io-smoke.md
dotnet run --project tests/RoyalTerminal.Benchmarks/RoyalTerminal.Benchmarks.csproj -c Release -- --skip-render --io --io-mode both --io-repeats 2 --fixture-size-mb 1 --fixtures /tmp/royalterminal-io-fixtures-better-smoke --output /tmp/royalterminal-io-better-smoke.md
dotnet run --project tests/RoyalTerminal.Benchmarks/RoyalTerminal.Benchmarks.csproj -c Release -- --skip-render --io --io-mode managed-vt --io-repeats 3 --fixture-size-mb 32 --fixtures /tmp/royalterminal-io-compare-fixtures --output /tmp/royalterminal-io-managed-vt-optimized.md
dotnet run --project tests/RoyalTerminal.Benchmarks/RoyalTerminal.Benchmarks.csproj -c Release -- --skip-render --vt-parse --io-repeats 5 --fixture-size-mb 16 --fixtures /tmp/royalterminal-io-profile-fixtures --output /tmp/royalterminal-vt-parse-after.md
dotnet run --project tests/RoyalTerminal.Benchmarks/RoyalTerminal.Benchmarks.csproj -c Release -- --skip-render --io --io-mode both --io-repeats 5 --fixture-size-mb 16 --fixtures /tmp/royalterminal-io-profile-fixtures --output /tmp/royalterminal-io-profile-after.md
dotnet run --project tests/RoyalTerminal.Benchmarks/RoyalTerminal.Benchmarks.csproj -c Release -- --skip-render --io --io-mode managed-vt --io-repeats 5 --fixture-size-mb 16 --fixtures /tmp/royalterminal-io-profile-fixtures --ghostty --ghostty-app /Users/wieslawsoltes/GitHub/RoyalTerminal/external/ghostty/macos/build/ReleaseLocal/Ghostty.app --output /tmp/royalterminal-ghostty-compare-16mb.md
dotnet run --project tests/RoyalTerminal.Benchmarks/RoyalTerminal.Benchmarks.csproj -c Release -- --skip-render --io --io-mode managed-vt --io-repeats 3 --fixture-size-mb 150 --fixtures /tmp/royalterminal-ghostty-150mb-fixtures --ghostty --ghostty-app /Users/wieslawsoltes/GitHub/RoyalTerminal/external/ghostty/macos/build/ReleaseLocal/Ghostty.app --output /tmp/royalterminal-ghostty-compare-150mb.md
dotnet test tests/RoyalTerminal.Tests/RoyalTerminal.Tests.csproj -c Release --filter "FullyQualifiedName~UnicodeWidthTests|FullyQualifiedName~TerminalScreenTests"
git diff --check

Results:

• Full test suite: 1238 passed, 16 skipped.
• PTY NativeAOT smoke publish and binary run: passed.
• Demo NativeAOT publish for osx-arm64: passed.
• IO smoke benchmark: passed; all fixture scenarios produced 64 KiB largest batches.
• IO both-mode smoke benchmark: passed.
• IO managed-VT 32 MiB benchmark: passed.
• Managed VT parser profile: passed.
• PTY IO profile after parser/scroll optimization: passed.
• Installed Ghostty 16 MiB comparison: passed.
• Installed Ghostty 150 MiB comparison: passed.
• Focused terminal screen and Unicode width tests: passed, 87 tests.
• Whitespace check: clean.

The demo NativeAOT publish emits macOS linker debug-info module-cache warnings from the toolchain, but no trim or AOT analysis errors.

Main vs Optimized PTY IO Comparison

The earlier managed-VT numbers were off because the benchmark was mixing PTY dispatch shape with managed parser/screen allocation cost. Parser-only profiling showed the real bottleneck: whole-screen scrolling allocated a fresh TerminalRow for every new line even after scrollback was full, and printable ASCII took the full Unicode grapheme/category and width path per codepoint.

Parser-only BasicVtProcessor.Process before/after on 16 MiB fixtures, five repeats:

Scenario	Before MiB/s	After MiB/s	Speedup	Before alloc/MiB	After alloc/MiB	Allocation reduction
ASCII	15.359	76.998	5.01x	49,923,344 B	3,045,861 B	16.4x
Unicode	19.117	60.862	3.18x	37,642,356 B	5,801,346 B	6.5x
CSI	81.661	114.322	1.40x	1,025 B	1,025 B	1.0x

Compared origin/main (97d4c4d) against this branch (f87b354) on macOS arm64 with the same 16 MiB fixtures and five repeats. The benchmark runs /usr/bin/time -p cat through a real PTY, feeds every delivered batch through BasicVtProcessor, records terminal-side elapsed throughput, records child cat wall time, and reports the median run.

Scenario	Main terminal MiB/s	Optimized terminal MiB/s	Terminal speedup	Main child MiB/s	Optimized child MiB/s	Child speedup	Main batches	Optimized batches	Batch reduction
ASCII	13.778	64.584	4.69x	13.913	66.667	4.79x	16,590	445	37.3x
Unicode	17.609	50.040	2.84x	17.778	51.613	2.90x	16,567	408	40.6x
CSI	60.073	89.657	1.49x	61.538	94.118	1.53x	16,627	511	32.5x

This now improves both sides of the original problem: saturated PTY output is delivered as 64 KiB batches instead of serial 1 KiB dispatches, and the managed parser no longer spends most of its time allocating rows for steady-state scrolling.

Installed Ghostty Comparison

Compared this branch against installed Ghostty Ghostty 1.3.2-HEAD+28972454c from /Users/wieslawsoltes/GitHub/RoyalTerminal/external/ghostty/macos/build/ReleaseLocal/Ghostty.app. Both columns use writer-side /usr/bin/time -p cat wall time, so this compares how quickly the child process can write the same fixture into each terminal path. RoyalTerminal used --io-mode managed-vt.

16 MiB fixtures, five repeats:

Scenario	RoyalTerminal MiB/s	Ghostty MiB/s	Royal/Ghostty	Royal real (ms)	Ghostty real (ms)
ASCII	61.538	53.333	1.154x	260	300
Unicode	47.059	45.714	1.029x	340	350
CSI	80.000	39.024	2.050x	200	410

150 MiB fixtures, three repeats:

Scenario	RoyalTerminal MiB/s	Ghostty MiB/s	Royal/Ghostty	Royal real (ms)	Ghostty real (ms)
ASCII	68.807	78.534	0.876x	2180	1910
Unicode	56.180	70.093	0.801x	2670	2140
CSI	86.705	62.241	1.393x	1730	2410

Interpretation: this RoyalTerminal branch is competitive with the installed Ghostty build on writer-side throughput. Ghostty is ahead on larger ASCII and Unicode fixtures, while RoyalTerminal is ahead on the CSI-heavy fixture. This is not a renderer frame benchmark and does not compare GPU presentation latency.

Follow-Up

• Extend the external-terminal comparison to Alacritty, Kitty, and other terminals with identical shell/render settings and hardware.
• Continue profiling deeper VT parser paths after the read-side and steady-state scroll allocation bottlenecks.
• Add CI jobs for PTY NativeAOT smoke and demo NativeAOT publish on supported RIDs.
• Revisit the optional Pretext path when the dependency exposes trim/AOT-safe metadata.