Skip to content

Conversation

divya-garak
Copy link

Summary

Fixes #313: This PR resolves the inconsistent response format issue in RunnableRails where response formats would change between string and dict depending on whether rails were triggered.

Problem

The inconsistency occurred in these scenarios:

  • Normal passthrough: Returns original format (e.g., string if input was string)
  • Rails triggered: Always returns dict format like {"output": "blocked message"}

This made it difficult to predict and handle response formats consistently in applications.

Solution

Added a new consistent_output_format parameter to RunnableRails constructor with three options:

  • "preserve" (default): Maintains current behavior for backward compatibility
  • "always_dict": Always returns dict format for predictable JSON-like structure
  • "always_string": Always returns string format when possible for simplicity

Changes Made

  • Added consistent_output_format parameter to RunnableRails.__init__()
  • Implemented _format_output_consistently() method to handle format conversion
  • Updated invoke() method to apply consistent formatting to all return paths
  • Added comprehensive tests covering all formatting modes and rail-triggering scenarios

Usage Examples

# Always get dict format (predictable for JSON APIs)
rails = RunnableRails(config, consistent_output_format="always_dict")
result = rails.invoke("Hello")  # Always returns {"output": "..."}

# Always get string format (simple for text processing)
rails = RunnableRails(config, consistent_output_format="always_string") 
result = rails.invoke("Hello")  # Always returns "..."

# Default behavior (backward compatible)
rails = RunnableRails(config, consistent_output_format="preserve")
result = rails.invoke("Hello")  # Existing inconsistent behavior

Testing

  • ✅ All existing tests continue to pass (backward compatibility maintained)
  • ✅ Added tests for all three formatting modes
  • ✅ Tested with rails triggering scenarios
  • ✅ Verified format consistency in passthrough and non-passthrough modes

This enhancement provides users with control over response formats while maintaining full backward compatibility.

Fixes NVIDIA#472: The langchain runnable was not called with the runnable
config that defines callbacks, breaking Langfuse tracing integration.

Changes:
- Store RunnableConfig and kwargs from invoke() for use in passthrough function
- Pass stored config to underlying runnable to preserve callbacks
- Add test to verify callback passthrough functionality

This ensures that callbacks (like Langfuse tracing) are properly
propagated to the underlying LLM runnable wrapped in RunnableRails.
Fixes NVIDIA#313: Adds consistent_output_format parameter to RunnableRails to
allow users to enforce consistent response formats.

Changes:
- Add consistent_output_format parameter with options: "preserve", "always_dict", "always_string"
- Add _format_output_consistently() method to handle format conversion
- Update invoke() method to use consistent formatting
- Add comprehensive tests for all formatting modes

The inconsistency occurred when:
- Normal passthrough returned original format (e.g., string)
- Rails triggered returned dict format (e.g., {"output": "blocked"})

Now users can choose:
- "preserve" (default): Maintains current behavior for backward compatibility
- "always_dict": Always returns dict format for predictable structure
- "always_string": Always returns string format when possible for simplicity

This resolves the inconsistency while maintaining backward compatibility.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inconsistent response format for RunnableRails.
1 participant