Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optional content field in Chat Completions Request #3021

Draft
wants to merge 2 commits into
base: git_v2.4.1
Choose a base branch
from

Conversation

saileshd1402
Copy link

@saileshd1402 saileshd1402 commented Feb 13, 2025

Currently, TGI is validating that "content" field always exists in chat completions request. But in case of an "assistant" message type in OpenAI messages struct, content can be empty.

According to OpenAI spec: "The contents of the assistant message. Required unless tool_calls or function_call is specified."

Example:

{
    "model": "llama-3-1-8b",
    "stream": false,
    "messages": [
        {
            "role": "assistant",
            "tool_calls": [
                {
                    "id": "0",
                    "function": {
                        "arguments": "{\"location\":\"Boston\",\"unit\":\"celsius\"}",
                        "name": "get_current_weather"
                    },
                    "type": "function"
                }
            ]
        }
    ]
}

Error returned by TGI:

Failed to deserialize the JSON body into the target type: messages[0]: missing field `content` at line 17 column 9

This PR fixes that by making content field optional and adding tool_calls to the Message struct (used in ChatRequest)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant