XNNPACK EP: concurrent InferenceSession creation races the lazily-created stored allocator (use-after-free)

[christian-apollo]

Describe the issue

The XNNPACK execution provider keeps a single process-wide "stored allocator" that it creates lazily behind an unsynchronised null check. When two InferenceSessions are created concurrently (each getting its own XnnpackExecutionProvider) before any XNNPACK session has been created in the process, the two CreatePreferredAllocators() calls race, one allocator is freed while XNNPACK is left holding a pointer into it, and every subsequent XNNPACK allocation/free in the process is a use-after-free.

The relevant code:

onnxruntime/core/providers/xnnpack/xnnpack_execution_provider.cc:

std::vector<AllocatorPtr> XnnpackExecutionProvider::CreatePreferredAllocators() {
  const auto& [stored_allocator, xnn_allocator] = GetStoredAllocator();
  if (!stored_allocator) {                       // <-- unsynchronised read
    const AllocatorCreationInfo allocator_info(...);
    stored_allocator = CreateAllocator(allocator_info);  // <-- unsynchronised write to a shared static
  }
  xnn_allocator->context = stored_allocator.get();       // <-- shared static mutated by both threads
  const xnn_status st = xnn_initialize(xnn_allocator);   // XNNPACK copies the pointer once (XNN_INIT_ONCE)
  ...
}

onnxruntime/core/providers/xnnpack/xnnpack_init.cc:

std::pair<AllocatorPtr&, xnn_allocator*> GetStoredAllocator() {
  static AllocatorPtr ort_allocator;                 // shared across all EP instances
  static xnn_allocator xnn_allocator_wrapper_ = { ort_allocator.get(), ... };
  return {ort_allocator, &xnn_allocator_wrapper_};
}

Two threads can both observe !stored_allocator, both CreateAllocator, and the second assignment releases the first shared_ptr. Because xnn_initialize records the allocator only on the first call (XNN_INIT_ONCE / a CAS on init_allocator), XNNPACK can end up permanently pointing at the IAllocator owned by the freed shared_ptr. From then on, xnn_allocate / xnn_aligned_allocate / xnn_deallocate dereference a freed IAllocator*.

Observed crash signature (Apple, symbolicated): EXC_BAD_ACCESS in onnxruntime::xnnpack::(anonymous namespace)::xnn_aligned_allocate at the virtual IAllocator::Alloc call through the context pointer. Because the corrupted pointer lives for the life of the process, the fault surfaces in whichever XNNPACK op runs next, most visibly:

• session creation, in Gemm::PrePack / MatMul::PrePack -> xnn_create_fully_connected_nc_f32 -> xnn_aligned_allocate
• inference, in reshape_fully_connected_nc
• session release, in xnn_destroy_operator -> xnn_deallocate

To reproduce

On a fresh process, create several sessions on separate threads at once, each appending the XNNPACK EP, e.g.:

Ort::Env env(ORT_LOGGING_LEVEL_ERROR, "race");
std::vector<std::thread> pool;
std::vector<std::shared_ptr<Ort::Session>> sessions(N);
for (int t = 0; t < N; ++t) {
  pool.emplace_back([&, t]{
    Ort::SessionOptions so;
    so.AppendExecutionProvider("XNNPACK", {{"intra_op_num_threads", "1"}});
    // line all threads up so construction starts simultaneously, then:
    sessions[t] = std::make_shared<Ort::Session>(env, model_path, so);
  });
}
for (auto& th : pool) th.join();

Reproduced deterministically on macOS arm64 (ORT 1.27.0) with MallocScribble=1 so the freed allocator faults immediately: with N=8 it crashes ~40-60% of runs; serialising session construction gives 0 crashes across hundreds of runs. Without heap poisoning the use-after-free is latent (reads stale-but-valid memory) and surfaces intermittently, which matches how it appears in the field. The same code is present in 1.29.0 and on main.

Urgency

Moderate. It is intermittent and only affects concurrent first-time session creation, but when it triggers it corrupts a process-global pointer and crashes unpredictably for the rest of the process's life.

Platform

Mac (reproduced), also affects iOS and Android (shared code).

OS Version

macOS 15 (arm64); observed in production on iOS.

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.27.0 (also 1.29.0 and current main)

ONNX Runtime API

C++

Architecture

ARM64

Execution Provider

Other / possibly contributes (XNNPACK)

Suggested fix

Make the lazy creation of the stored allocator (and the xnn_allocator->context write + xnn_initialize) thread-safe, e.g. a std::call_once / mutex inside GetStoredAllocator() or around the init block in CreatePreferredAllocators(). As a consumer-side workaround, serialising Ort::Session construction with a process-wide mutex avoids it.

[skottmckay]

Urgency
Moderate. It is intermittent and only affects concurrent first-time session creation, but when it triggers it corrupts a process-global pointer and crashes unpredictably for the rest of the process's life.

Is there a production use-case where you would create multiple sessions concurrently at exactly the same time?

[christian-apollo]

Yes, and it is not an exotic one. Ours is an openWakeWord pipeline in a React Native app: three small models (melspectrogram, embedding, keyword head) that are only useful as a set, so they are loaded together when the feature starts:

const [mel, emb, kw] = await Promise.all([
  InferenceSession.create(melPath, {executionProviders: ['xnnpack', 'cpu'], ...}),
  InferenceSession.create(embPath, ...),
  InferenceSession.create(kwPath, ...),
]);

onnxruntime-react-native runs each loadModel on its own std::thread (LoadModelAsyncWorker in cpp/InferenceSessionHostObject.cpp, via AsyncWorker), so three create calls awaited together are three Ort::Session constructors running at the same time, and on a cold process they all race the first-time XNNPACK init. That is the crash we see on iOS in production. Nothing in the JS API suggests the calls need to be serialised, and Promise.all over model loads is the idiomatic way to write it, so I would expect anyone with a multi-model pipeline on the RN binding (encoder + decoder, detector + classifier, VAD + ASR) to have the same shape.

It only bites on the very first XNNPACK session in the process, which is why it presents as an intermittent startup crash rather than something anyone reproduces on demand.

Serialising session creation on our side is an easy workaround, so this is not blocking us. But the shared state is already meant to be a process singleton, the fix is a call_once or mutex around it, and the failure mode is a silently corrupted process-global pointer, so I think it is worth taking even if concurrent creation is uncommon in the C++ API.

[SiluPanda]

I'm working on a focused fix for this. I rechecked the current implementation and did not find a linked fix or implementation claim. The plan is to synchronize the process-wide allocator creation, wrapper context publication, and XNNPACK initialization together, and add concurrent cold-start regression coverage.

I'll validate on macOS arm64 (CPU/XNNPACK; no GPU is needed for this path) and report exactly what I can reproduce and test. I'll share a PR here when the patch and tests are ready.

[SiluPanda]

christian-apollo I've opened a ready-for-review fix: #32489.

The new cold-start allocator regression failed in 12/20 runs with the original implementation, then passed 100/100 with the fix on macOS arm64 under MallocScribble=1. All 20 enabled XNNPACK tests also passed locally.

Would you be able to validate the patch with your original macOS reproducer and/or the three-model React Native/iOS startup workload, with concurrent model loading restored? My regression exercises the allocator initialization and lifetime directly; I have not tested your complete application workflow. No GPU is needed for this XNNPACK path. Thank you for the detailed report and production context.