docs/concepts/structured_outputs/

[giscus[bot]]

docs/concepts/structured_outputs/

Overview

https://python.langchain.com/docs/concepts/structured_outputs/

[SharoMonk]

model = ChatGroq(model="llama3-8b-8192")
model_with_structured_output = model.with_structured_output(schema=ResponseFormatter)

Invoke the model to produce structured output that matches the schema

structured_output = model_with_structured_output.invoke(
"My question is why did you do it and how was it done?"
)
print(structured_output)
answer='Why did you do it?' followup_question='How was it done?'

[passaadellism]

Can we return the response metadata that are contained to the simple invoke eg (AIMessage) with the structured schema?

response_metadata={'token_usage': {'completion_tokens': 107, 'prompt_tokens': 409, 'total_tokens': 516, 'completion_tokens_details': {'reasoning_tokens': 0, 'audio_tokens': 0, 'accepted_prediction_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'cached_tokens': 0, 'audio_tokens': 0}}, 'model_name': 'gpt-4o-2024-08-06', 'system_fingerprint': 'fp_xxxxxxxx', 'finish_reason': 'stop', 'logprobs': None}, id='run-xxxxxxxxxxxxxx', usage_metadata={'input_tokens': 409, 'output_tokens': 107, 'total_tokens': 516}

[syntaxtrash]

I'm not sure if I understand your question. Are you asking how to get the response_metadata attribute from AIMessage?

You can access it directly like this:
response.response_metadata

[Arghya721]

Using a prompt template along with tool calling is giving me an error. Also it seems the llm is not able to read the input prompt provided.

import dotenv
import os
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.messages import HumanMessage, SystemMessage
from pydantic import BaseModel, Field

dotenv.load_dotenv()

class ResponseFormatter(BaseModel):
    """Always use this tool to structure your response to the user."""
    language: str = Field(description="The language of the translation")
    text: str = Field(description="The text to be translated")
    translation: str = Field(description="The translation of the text")

open_ai = ChatOpenAI(
    model="gpt-4o-mini"
)

open_ai_tools = open_ai.bind_tools([ResponseFormatter])

system_template = """
Translate the following from English into {language}:
"""

prompt_template = ChatPromptTemplate.from_messages(
    [("system", system_template), ("user", "{text}")]
)

prompt = prompt_template.invoke({"language" : "German", "text" : "hi!"})

print(prompt)

ai_msg = open_ai_tools.invoke(prompt)

print(ai_msg)

print(ai_msg.tool_calls[0]["args"])

pydantic_object = ResponseFormatter.model_validate(ai_msg.tool_calls[0]["args"])

print(pydantic_object)

Output

messages=[SystemMessage(content='\nTranslate the following from English into German:\n', additional_kwargs={}, response_metadata={}), HumanMessage(content='hi!', additional_kwargs={}, response_metadata={})]
content='Hello! How can I assist you today?' additional_kwargs={'refusal': None} response_metadata={'token_usage': {'completion_tokens': 11, 'prompt_tokens': 87, 'total_tokens': 98, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_06737a9306', 'finish_reason': 'stop', 'logprobs': None} id='run-a67cb852-ad94-4923-95a1-89535e8f6591-0' usage_metadata={'input_tokens': 87, 'output_tokens': 11, 'total_tokens': 98, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}}
Traceback (most recent call last):
  File "D:\langchain tutorial\structured_output.py", line 40, in <module>
    print(ai_msg.tool_calls[0]["args"])
          ~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range