Add a session config to share pre-packed weights of all initializers across sessions

[Yoonseok Kim (yoonseok-kim)]

Description

Initializers registered via OrtApi::AddInitializer — weights the user supplies as in-memory buffers — already share their pre-packed copies across sessions through the prepacked weights container. Weights that come from the model itself — a file path, a byte buffer, or external data — do not: they are pre-packed per session even when a container is attached.

This adds an opt-in session config, session.share_prepacked_weights_for_all_initializers. When set to "1" and a container is attached, every constant initializer on the CPU EP enrolls in the container, so sessions over the same model reuse one pre-packed copy. The default ("0") leaves behavior unchanged.

Kernels that pre-pack without producing shareable buffers (e.g. fp16 LayerNormalization) are skipped for initializers enrolled by this option; AddInitializer keeps the existing strict check.

Measured with a 64 MiB fp32 MatMul weight and 8 sessions sharing one container: per-session resident memory drops from ~64 MiB to ~40 kB, with identical outputs.

Added tests: cross-session reuse without AddInitializer, and the kernel-owned pre-pack case with the option enabled.

Motivation and Context

In a session pool loading weights from the model file, pre-packed copies scale with the session count. The only workaround is re-registering every initializer through AddInitializer with user-supplied buffers, which requires parsing the model and keeping the buffers alive, and defeats mmap'd external weights. #15301 raised the same problem.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟢 Approval recommended

The behavior change is gated behind a new opt-in config key, is covered by targeted unit tests, and leaves default behavior unchanged.

Pull request overview

This PR adds an opt-in SessionOptions config key that allows model-owned constant initializers (not just user-registered ones via OrtApi::AddInitializer) to enroll in an attached PrepackedWeightsContainer, enabling cross-session reuse of CPU EP pre-packed weights for those initializers.

Changes:

• Introduces session.share_prepacked_weights_for_all_initializers session config key and documents its behavior.
• Extends SessionState::PrepackConstantInitializedTensors to optionally enroll all constant initializers (CPU EP only) into the shared prepacked-weights container.
• Adds unit tests validating cross-session reuse and confirming that kernel-owned (non-shareable) pre-packs are tolerated (no failure) under the new option.

File summaries

File	Description
onnxruntime/test/framework/session_state_test.cc	Adds tests for cross-session sharing of model-owned initializers and for skipping strict enforcement on kernel-owned pre-packs when the new option is enabled.
onnxruntime/core/framework/session_state.cc	Adds opt-in logic to enroll all constant initializers into the shared prepacked-weights container and relaxes the strict “must provide shareable buffers” check for share-all-enrolled initializers.
include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h	Adds and documents the new public session config key string.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.