Skip to content

Conversation

Pouyanpi
Copy link
Collaborator

@Pouyanpi Pouyanpi commented Sep 3, 2025

Ensure AIMessage responses from RunnableRails contain the same metadata fields (response_metadata, usage_metadata, additional_kwargs, id) as direct LLM calls, enabling consistent LangChain integration behavior.

@Pouyanpi Pouyanpi force-pushed the feat/runnable-rails-ai-message branch from 8309931 to 8e1033c Compare September 3, 2025 09:16
@Pouyanpi Pouyanpi added this to the v0.17.0 milestone Sep 3, 2025
@Pouyanpi Pouyanpi added enhancement New feature or request runnable labels Sep 3, 2025
@Pouyanpi Pouyanpi self-assigned this Sep 3, 2025
@Pouyanpi Pouyanpi force-pushed the feat/runnable-rails-ai-message branch 2 times, most recently from 1c5837e to 6a6050b Compare September 3, 2025 10:29
@Pouyanpi Pouyanpi force-pushed the feat/runnable-rails branch from 7ae2f3a to 68f438e Compare September 9, 2025 09:22
@Pouyanpi Pouyanpi force-pushed the feat/runnable-rails-ai-message branch 2 times, most recently from ce4d6c1 to 0b114ba Compare September 9, 2025 09:40
@Pouyanpi Pouyanpi requested a review from Copilot September 9, 2025 09:42
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements AIMessage metadata parity in RunnableRails by ensuring that LLM metadata (response_metadata, usage_metadata, additional_kwargs, id) from the underlying LLM is preserved and passed through to the final AIMessage responses. This enables consistent behavior when using RunnableRails within LangChain chains.

  • Adds metadata preservation infrastructure to capture and store LLM response metadata
  • Updates RunnableRails output formatting to include preserved metadata in AIMessage responses
  • Adds comprehensive test coverage for metadata preservation scenarios

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/runnable_rails/test_runnable_rails.py Adds integration test to verify metadata preservation through RunnableRails
tests/runnable_rails/test_metadata.py Comprehensive test suite for metadata preservation with various input/output formats
nemoguardrails/rails/llm/options.py Adds llm_metadata field to GenerationResponse to store LLM metadata
nemoguardrails/rails/llm/llmrails.py Updates to include llm_metadata in generation responses
nemoguardrails/integrations/langchain/runnable_rails.py Updates output formatting methods to include metadata in AIMessage creation
nemoguardrails/context.py Adds context variable for storing LLM response metadata
nemoguardrails/actions/llm/utils.py Adds function to store and retrieve LLM response metadata

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@Pouyanpi Pouyanpi marked this pull request as ready for review September 10, 2025 12:47
@Pouyanpi Pouyanpi changed the title feat(runnable_rails): implement AIMessage metadata parity in RunnableRails feat(runnable-rails): implement AIMessage metadata parity in RunnableRails Sep 10, 2025
@Pouyanpi Pouyanpi force-pushed the feat/runnable-rails branch 3 times, most recently from feb4447 to 0c2a65e Compare September 15, 2025 09:32
…Rails

Ensure AIMessage responses from RunnableRails contain the same metadata
fields (response_metadata, usage_metadata, additional_kwargs, id) as
direct LLM calls, enabling consistent LangChain integration behavior.
@Pouyanpi Pouyanpi force-pushed the feat/runnable-rails-ai-message branch from 0b114ba to 0e0ea4f Compare September 15, 2025 09:38
@Pouyanpi Pouyanpi requested a review from tgasser-nv September 15, 2025 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request runnable
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant