Fix concurrent XNNPACK allocator initialization

[Silu Panda (SiluPanda)]

Description

Make the process-wide XNNPACK allocator initialization thread-safe with std::call_once. The once callback covers allocator creation, wrapper context publication, and xnn_initialize, so concurrent EPs cannot replace the allocator or rewrite the context while existing kernels use it.

Initialization errors still throw inside the callback, allowing a retry. The existing stored-allocator guard is retained so a retry does not replace an allocator XNNPACK may already have retained.

Add a cold-start regression test that re-executes in a fresh subprocess, synchronizes 32 calls to CreatePreferredAllocators(), and checks allocator identity across EPs. After releasing the per-EP owners, it exercises XNNPACK's retained allocator through workspace allocation/release and verifies subsequent EP creation can still use the same allocator.

Motivation and Context

Fixes #32461.

XNNPACK retains its allocator context process-wide. Concurrent first-time session initialization could replace the owning shared_ptr after XNNPACK captured its pointer, resulting in a dangling context. The wrapper is also read by existing kernels, so successful initialization must not rewrite it on subsequent EP creation.

Validation

Locally built on macOS 26.3.1 arm64 with Apple Clang 17, Release, and XNNPACK enabled; no GPU required.

• Negative control: with the new test but the original provider implementation, 12 of 20 fresh-process runs failed under MallocScribble=1 (including 7 SIGSEGV failures).
• Fixed regression: 100 of 100 fresh-process runs passed under MallocScribble=1.
• XNNPACK suites: all 20 enabled tests selected by Xnnpack* passed; 12 existing tests remain disabled.
• Pinned lintrunner / clang-format checks and git diff --check passed.

Build configuration:

python tools/ci_build/build.py --config Release --build_dir build/MacOS \
  --update --build_shared_lib --use_xnnpack --cmake_generator Ninja \
  --skip_submodule_sync --cmake_extra_defines \
  CMAKE_OSX_ARCHITECTURES=arm64 FETCHCONTENT_TRY_FIND_PACKAGE_MODE=NEVER
python tools/ci_build/build.py --config Release --build_dir build/MacOS \
  --build --build_shared_lib --use_xnnpack --parallel 5 \
  --target onnxruntime_provider_test

The pinned-dependency option avoids a locally installed Protobuf/runtime mismatch. When building only the provider-test target, I copied onnxruntime/test/testdata and samples to the build output directory, matching the fixture-copy steps normally attached to onnxruntime_test_all.

From build/MacOS/Release:

MallocScribble=1 ./onnxruntime_provider_test \
  --gtest_filter=XnnpackEPDeathTest.ConcurrentAllocatorInitialization --gtest_repeat=100
MallocScribble=1 ./onnxruntime_provider_test --gtest_filter='Xnnpack*'

The regression directly exercises allocator initialization/lifetime, not the reporter's complete React Native workload. iOS/React Native and native Windows have not been tested locally. The subprocess test is guarded by GTEST_HAS_DEATH_TEST and is not compiled on iOS/WASM. Initialization-failure retry behavior was reviewed but not fault-injection tested.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Silu Panda (SiluPanda)]

Scott McKay (@skottmckay) could you please review this focused fix for #32461, given your context on the issue and the XNNPACK allocator implementation?

The new fresh-process regression failed 12/20 times with the original implementation and passed 100/100 with the fix on macOS arm64 (MallocScribble=1). All 20 enabled XNNPACK tests pass locally. I have also asked the reporter to validate their original React Native/iOS startup workload.

Could you enable the appropriate PR validation when eligible, particularly the native macOS/Windows XNNPACK coverage? The CLA bot has requested a signature, which I will leave to the contributor to complete.

[Copilot]

🟢 Approval recommended

The synchronization correctly protects the full initialization sequence, and the regression test exercises the original race and allocator lifetime.

Pull request overview

Makes process-wide XNNPACK allocator initialization thread-safe and preserves allocator lifetime.

Changes:

• Uses std::call_once for allocator creation, publication, and XNNPACK initialization.
• Adds a fresh-process, 32-thread regression test covering identity and lifetime.

File summaries

File	Description
onnxruntime/core/providers/xnnpack/xnnpack_execution_provider.cc	Serializes global allocator initialization.
onnxruntime/test/providers/xnnpack/xnnpack_basic_test.cc	Tests concurrent initialization and retained allocator use.

No actionable findings identified.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Silu Panda (SiluPanda)]

@microsoft-github-policy-service agree

@microsoft-github-policy-service agree

[christian-apollo]

Validated the patch at cdc43f6 on both the original macOS reproducer and the React Native / iOS workload from #32461, with concurrent model loading restored. It fixes the race.

macOS arm64 (macOS 26, Apple clang 21, Release, --build_shared_lib --use_xnnpack, Ninja, MallocScribble=1). Fix vs. the same tree with only the provider change reverted (test kept):

	with fix	provider change reverted
XnnpackEPDeathTest.ConcurrentAllocatorInitialization, --gtest_repeat=100	100/100 pass	62/100 fail
Xnnpack* suite	20/20 pass, 12 disabled	not run
my reproducer from the issue, 8 threads x 150 runs, per model	0 crashes	1 crash per model (SIGSEGV / SIGBUS)
my reproducer, the app's 3-model shape, 300 runs	0 crashes	0 crashes

For reference, the same reproducer crashes 49/100 against the released 1.27.0 binary we ship, so it is a much weaker control against main than your death test. Your test is the one that matters there and its failure rate without the fix matches your numbers.

iOS, React Native. Built the iOS static framework from the branch (--ios --use_xnnpack --use_coreml --build_apple_framework), swapped it into the onnxruntime-c pod of our app, removed our session-creation mutex, and ran on an iPhone 17 Pro with MallocScribble=1:

• stress harness, 8 / 32 / 64 concurrent InferenceSession.create per cold launch, each session run once then released: 28/28 launches clean;
• the real wake-word engine, three models created with Promise.all: 5/5 cold launches came up with all three sessions live and kept running inference.

One caveat on the device numbers: our unfixed build never crashed from that JS harness either (0/28 today, 0/30 last week), even though scribbling is confirmed active in the process. Without a spin barrier the RN loader threads rarely land in the window, so treat the iOS runs as a functional check of concurrent creation against the fixed library, not as a crash-versus-no-crash comparison. The macOS death test is the negative control.

Also checked that moving xnn_initialize under call_once loses nothing: the pinned XNNPACK latches the allocator on the first call anyway (XNN_INIT_ONCE), and its xnn_deinitialize, which the EP destructor calls, is an empty function, so repeated init calls after the first were already no-ops.

Two small notes on the iOS build, in case it helps the CI story: with CMake 4 the CoreML path needs CMAKE_POLICY_VERSION_MINIMUM=3.5 (psimd's old cmake_minimum_required), and the death test is correctly excluded there since GTEST_HAS_DEATH_TEST is off on iOS.

Thanks for the quick turnaround on this.