-
Notifications
You must be signed in to change notification settings - Fork 55
Enable Session Option to Stop Context Sharing #822
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: ovep-develop
Are you sure you want to change the base?
Conversation
The initial implementation of Clear() did not reset the name of the external metadata file and causes the new context from using the same name as the previous context.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM !
@n1harika .. We can merge when you have a unit test ready for this. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements a new session option StopShareEpContexts
for the OpenVINO execution provider that enables explicit cleanup of globally shared EP contexts after a session completes. This provides controlled cleanup of shared weights, metadata, and other cached artifacts.
- Adds a new session option
so_stop_share_ep_contexts
to control context cleanup behavior - Implements cleanup logic in the
SharedContext
class with aclear()
method - Triggers context cleanup in the execution provider's
Compile
method when the option is enabled
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
File | Description |
---|---|
openvino_provider_factory.cc | Parses the new StopShareEpContexts session option from configuration |
openvino_execution_provider.cc | Adds shared context cleanup in destructor and compile method based on the new option |
contexts.h | Defines the clear() methods for SharedContext and its nested structures |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"delete mapped_weights.release();"
This might need a change as you are deleting a pointer you are releasing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
onnxruntime/core/providers/openvino/openvino_execution_provider.cc
Outdated
Show resolved
Hide resolved
…r.cc Co-authored-by: Copilot <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks Good To Merge !
This PR enables the session option: StopShareEpContexts ("ep.stop_share_ep_contexts" = 1/0), to explicitly clear the globally shared EP context after a session completes. This allows controlled cleanup of shared weights, metadata, its file path, and other cached artifacts stored in SharedContext.
For example, for model abc and model xyz:
model_abc(ep.share_ep_contexts=1, ep.stop_share_ep_contexts=1)
After this session, all existing shared context data is cleared.
model_xyz(ep.share_ep_contexts=1)
This model starts with a fresh shared context, without any leftover metadata or weights from model_abc.