feat: Add instructions parameter in response object #3741

s-akhtar-baig · 2025-10-08T19:10:55Z

Problem

The current inline provider appends the user provided instructions to messages as a system prompt, but the returned response object does not contain the instructions field (as specified in the OpenAI responses spec).

What does this PR do?

This pull request adds the instruction field to the response object definition and updates the inline provider. It also ensures that instructions from previous response is not carried over to the next response (as specified in the openAI spec).

Closes #3566

Test Plan

Tested manually for change in model response w.r.t supplied instructions field.
Added unit test to check that the instructions from previous response is not carried over to the next response.
Added integration tests to check instructions parameter in the returned response object.
Added new recordings for the integration tests.

meta-cla · 2025-10-08T19:11:01Z

Hi @s-akhtar-baig!

Thank you for your pull request.

We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

meta-cla · 2025-10-08T19:20:55Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

ashwinb · 2025-10-10T16:38:16Z

docs/static/deprecated-llama-stack-spec.html

                        "type": "string",
                        "description": "(Optional) Truncation strategy applied to the response"
                    },
+                    "instructions": {


openai's docs say this can be a string or an array

Thanks, @ashwinb, for reviewing! I have updated the definition using openai/types/responses as my reference. Please take a look when you get a chance.

Interesting that this shows a String only but the Python client is definitely the source of truth!

FWIW, I tried this with the Python client talking directly to OpenAI (not Llama Stack):

r3 = open_ai_client.responses.create( model="gpt-4o", input="What is the capital of France?", instructions=["Always answer with rhyming poetry.", "Include the word 'cat' in your answer."] )

What I got was:

BadRequestError: Error code: 400 - {'error': {'message': "Invalid type for 'instructions': expected a string, but got an array instead.", 'type': 'invalid_request_error', 'param': 'instructions', 'code': 'invalid_type'}}

So the OpenAI client does indeed accept the list and send it off to the server but then the server rejected it because the server API expects a string only. So I guess the API reference that @leseb links to above is correct.

@jwm4, that makes sense! The link that @leseb shared brings me to the create endpoint which accepts string, but the response object can store instructions as a string or an array.

So is the conclusion that this PR is fine the way it is? (i.e., that Llama Stack should really just accept a string and not a list for instructions)

With these changes, we will be conforming to the OpenAI spec:

create response accepts instructions as a string

response object can store either a string or an array

instructions from previous response is not carried over

But, I am not sure whether that's what we want in Llama Stack or not. If we change create response to accept both a string and an array, then we won't be conforming to the spec and we don't have use-cases (that I can think of) to correctly implement logic that will handle array data type.

In [7]: from openai import OpenAI; client = OpenAI() In [8]: openai.__version__ Out[8]: '1.107.0' In [9]: client.responses.create.__annotations__ Out[9]: {'background': 'Optional[bool] | NotGiven', 'conversation': 'Optional[response_create_params.Conversation] | NotGiven', 'include': 'Optional[List[ResponseIncludable]] | NotGiven', 'input': 'Union[str, ResponseInputParam] | NotGiven', 'instructions': 'Optional[str] | NotGiven', 'max_output_tokens': 'Optional[int] | NotGiven', 'max_tool_calls': 'Optional[int] | NotGiven', 'metadata': 'Optional[Metadata] | NotGiven', 'model': 'ResponsesModel | NotGiven', 'parallel_tool_calls': 'Optional[bool] | NotGiven', 'previous_response_id': 'Optional[str] | NotGiven', 'prompt': 'Optional[ResponsePromptParam] | NotGiven', 'prompt_cache_key': 'str | NotGiven', 'reasoning': 'Optional[Reasoning] | NotGiven', 'safety_identifier': 'str | NotGiven', 'service_tier': "Optional[Literal['auto', 'default', 'flex', 'scale', 'priority']] | NotGiven", 'store': 'Optional[bool] | NotGiven', 'stream': 'Optional[Literal[False]] | Literal[True] | NotGiven', 'stream_options': 'Optional[response_create_params.StreamOptions] | NotGiven', 'temperature': 'Optional[float] | NotGiven', 'text': 'ResponseTextConfigParam | NotGiven', 'tool_choice': 'response_create_params.ToolChoice | NotGiven', 'tools': 'Iterable[ToolParam] | NotGiven', 'top_logprobs': 'Optional[int] | NotGiven', 'top_p': 'Optional[float] | NotGiven', 'truncation': "Optional[Literal['auto', 'disabled']] | NotGiven", 'user': 'str | NotGiven', 'extra_headers': 'Headers | None', 'extra_query': 'Query | None', 'extra_body': 'Body | None', 'timeout': 'float | httpx.Timeout | None | NotGiven', 'return': 'Response | Stream[ResponseStreamEvent]'}

yeah looks like the client (as mentioned by @s-akhtar-baig) only lists instructions as a str

s-akhtar-baig · 2025-10-13T19:12:12Z

llama_stack/providers/inline/agents/meta_reference/responses/openai_responses.py

+                # Omit instructions from previous response
+                previous_instructions = previous_response.instructions
+                if previous_instructions:
+                    assert isinstance(previous_instructions, str)


@ashwinb and @leseb, I'd appreciate your thoughts on this:

This condition will ensure that instructions is handled and stored as a string for the time being. Because the spec for create response specifies instructions as a string, and I am not sure what the use case would be for storing instructions as an array. Please let me know if I am missing anything.

My take on this is that if the spec say str | list, we should also update the create response.

Thanks, @leseb, for reviewing! I have handled other comments and added tests and recordings. Please take a look when you get a chance.

My take on this is that if the spec say str | list, we should also update the create response.

This makes sense! But the thing that comes to mind is that without use cases for other types, I am not sure if we can handle those types correctly, for the time being.

llama_stack/providers/inline/agents/meta_reference/responses/openai_responses.py

leseb · 2025-10-14T16:41:16Z

llama_stack/providers/inline/agents/meta_reference/responses/openai_responses.py

+                # Omit instructions from previous response
+                previous_instructions = previous_response.instructions
+                if previous_instructions:
+                    assert isinstance(previous_instructions, str)


My take on this is that if the spec say str | list, we should also update the create response.

llama_stack/providers/inline/agents/meta_reference/responses/openai_responses.py

jwm4

The logic here looks correct to me.

s-akhtar-baig · 2025-10-17T15:24:00Z

@ashwinb and @leseb, I have handled your comments and all the checks are passing now. Can you please review when you get a chance? Thanks!

ashwinb · 2025-10-18T00:47:56Z

llama_stack/apis/agents/openai_responses.py

    tools: list[OpenAIResponseTool] | None = None
    truncation: str | None = None
    usage: OpenAIResponseUsage | None = None
+    instructions: str | list[OpenAIResponseInput] | None = None


based on all the discussion so far, I think we should just do what you did first @s-akhtar-baig -- which is that this is just a simple str, that's it. this specific type is not good -- you really are not going to accept arbitrarily complex input as instructions here.

Thanks, @ashwinb!

The tests have passed, can you please merge when you get a chance? Thanks again!

s-akhtar-baig requested review from ashwinb, bbrowning, ehhuang, franciscojavierarceo, hardikjshah, leseb, mattf, raghotham, reluctantfuturist, slekkala1, terrytangyuan and yanxi0830 as code owners October 8, 2025 19:10

s-akhtar-baig changed the title ~~Add support for instructions parameter in response object~~ feat: Add support for instructions parameter in response object Oct 8, 2025

meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 8, 2025

s-akhtar-baig changed the title ~~feat: Add support for instructions parameter in response object~~ feat: Add instructions parameter in response object Oct 8, 2025

ashwinb requested changes Oct 10, 2025

View reviewed changes

s-akhtar-baig force-pushed the add_instructions_parameter branch from 5f503d6 to 2ce56ab Compare October 13, 2025 19:00

s-akhtar-baig commented Oct 13, 2025

View reviewed changes

leseb reviewed Oct 14, 2025

View reviewed changes

s-akhtar-baig force-pushed the add_instructions_parameter branch 2 times, most recently from 125aa75 to 3487d0a Compare October 14, 2025 21:39

jwm4 approved these changes Oct 15, 2025

View reviewed changes

s-akhtar-baig force-pushed the add_instructions_parameter branch 2 times, most recently from 5dc97a0 to a615f7b Compare October 16, 2025 21:44

s-akhtar-baig requested review from ashwinb and leseb October 16, 2025 21:48

ashwinb reviewed Oct 18, 2025

View reviewed changes

s-akhtar-baig and others added 7 commits October 20, 2025 15:34

add support for instructions parameter in response object

f176e1a

Recordings update from CI (suite: base)

dc26e67

Add unit test to check instructions with previous response instructions

41c55fa

Fix formatting issues

70bfe62

Resolve merge conflicts

f4556f1

Resolve merge conflicts

70217cc

Handle review comments

f7b6d7f

s-akhtar-baig force-pushed the add_instructions_parameter branch from a615f7b to f7b6d7f Compare October 20, 2025 19:39

ashwinb approved these changes Oct 20, 2025

View reviewed changes

ashwinb merged commit add64e8 into llamastack:main Oct 20, 2025
24 checks passed

feat: Add instructions parameter in response object #3741

feat: Add instructions parameter in response object #3741

Uh oh!

Conversation

s-akhtar-baig commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

What does this PR do?

Test Plan

Uh oh!

meta-cla bot commented Oct 8, 2025

Process

Uh oh!

meta-cla bot commented Oct 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

s-akhtar-baig Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

s-akhtar-baig Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jwm4 left a comment

Choose a reason for hiding this comment

Uh oh!

s-akhtar-baig commented Oct 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

s-akhtar-baig commented Oct 8, 2025 •

edited

Loading

s-akhtar-baig Oct 15, 2025 •

edited

Loading

franciscojavierarceo Oct 15, 2025 •

edited

Loading

s-akhtar-baig Oct 14, 2025 •

edited

Loading