Skip to content

Streaming Fails Due to {"include_usage": True} #442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
sixwaaaay opened this issue Apr 5, 2025 · 2 comments
Open

Streaming Fails Due to {"include_usage": True} #442

sixwaaaay opened this issue Apr 5, 2025 · 2 comments

Comments

@sixwaaaay
Copy link

Description
When using Mistral AI's OpenAI-compatible API with streaming output Runner.run_streamed, the hardcoded stream_options={"include_usage": True} in the code triggers a 422 Unprocessable Entity error. Mistral's API does not support the include_usage parameter in stream_options, causing compatibility issues.

Root Cause:

openai-agents-python/src/agents/models/openai_chatcompletions.py

Line 539 in 064e25b

stream_options={"include_usage": True} if stream else NOT_GIVEN,

Proposed Fix
Add an override mechanism for stream_options

Error

openai.UnprocessableEntityError: Error code: 422 - {'object': 'error', 'message': {'detail': [{'type': 'extra_forbidden', 'loc': ['body', 'stream_options', 'include_usage'], 'msg': 'Extra inputs are not permitted', 'input': True}]}, 'type': 'invalid_request_error', 'param': None, 'code': None}
@rm-openai
Copy link
Collaborator

Proposed fix makes sense, PR welcome!

@Ddper Ddper mentioned this issue Apr 10, 2025
Merged
@Ddper
Copy link
Contributor

Ddper commented Apr 10, 2025

Proposed fix makes sense, PR welcome!

@rm-openai I created a PR, pls review.

rm-openai pushed a commit that referenced this issue Apr 10, 2025
@Ddper
add overwrite mechanism for stream_options (#465)
8ded8a9 
fix issue #442

below is an example to overwrite include_usage


```
    result = Runner.run_streamed(
        agent,
        "Write a haiku about recursion in programming.",
        run_config=RunConfig(
            model_provider=CUSTOM_MODEL_PROVIDER,
            model_settings=ModelSettings(include_usage=True)
        ),
    )
```
Lightblues pushed a commit to Lightblues/openai-agents-python that referenced this issue Apr 13, 2025
@Ddper
add overwrite mechanism for stream_options (openai#465)
b1b7b44 
fix issue openai#442

below is an example to overwrite include_usage


```
    result = Runner.run_streamed(
        agent,
        "Write a haiku about recursion in programming.",
        run_config=RunConfig(
            model_provider=CUSTOM_MODEL_PROVIDER,
            model_settings=ModelSettings(include_usage=True)
        ),
    )
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Ddper @sixwaaaay @rm-openai