Skip to content

Conversation

@hntrl
Copy link
Member

@hntrl hntrl commented Dec 18, 2025

(breaking - do not merge)

ref langchain-ai/langchainjs#9688

  • Extracts and aggregates usage_metadata from generations, storing it in run.extra["metadata"]
  • Flattens output structure when there's a single generation with a message, simplifying traced output

Example

Before (nested structure):

run.outputs = {
    "generations": [
        [
            {
                "text": "Hello!",
                "message": {
                    "content": "Hello!",
                    "usage_metadata": {"input_tokens": 10, "output_tokens": 5}
                }
            }
        ]
    ]
}

After (flattened):

run.outputs = {
    "content": "Hello!",
    "usage_metadata": {"input_tokens": 10, "output_tokens": 5}
}

run.extra["metadata"]["usage_metadata"] = {"input_tokens": 10, "output_tokens": 5}

Flattening only occurs for single-generation outputs; multiple generations/batches retain the original nested structure.

@hntrl hntrl requested a review from eyurtsev as a code owner December 18, 2025 06:44
@github-actions github-actions bot added infra PRs made that include chores, devops, repo meta changes core `langchain-core` package issues & PRs and removed infra PRs made that include chores, devops, repo meta changes labels Dec 18, 2025
@codspeed-hq
Copy link

codspeed-hq bot commented Dec 18, 2025

CodSpeed Performance Report

Merging #34415 will improve performances by 14.91%

Comparing hunter/flatten-lct-generations (8178e9b) with hunter/usage-metadata-extras (9511665)

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

⚡ 5 improvements
✅ 8 untouched
⏩ 21 skipped1

Benchmarks breakdown

Mode Benchmark BASE HEAD Change
WallTime test_import_time[PydanticOutputParser] 597.5 ms 520 ms +14.91%
WallTime test_import_time[RunnableLambda] 545.1 ms 481.7 ms +13.17%
WallTime test_import_time[Runnable] 538.9 ms 486.6 ms +10.77%
WallTime test_import_time[CallbackManager] 513.1 ms 453 ms +13.27%
WallTime test_import_time[tool] 574.3 ms 509.6 ms +12.7%

Footnotes

  1. 21 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@hntrl hntrl force-pushed the hunter/flatten-lct-generations branch from 01f2bef to 4fa21f7 Compare December 18, 2025 18:51
@hntrl hntrl force-pushed the hunter/flatten-lct-generations branch from 4fa21f7 to 02ced19 Compare December 18, 2025 19:03
@hntrl hntrl requested a review from mdrxy December 18, 2025 19:29
@hntrl hntrl added the breaking Breaking changes label Dec 18, 2025
@hntrl hntrl marked this pull request as draft December 18, 2025 21:53
@hntrl
Copy link
Member Author

hntrl commented Dec 18, 2025

Punting to v2

@mdrxy mdrxy changed the title chore(core): flatten generations for LangChainTracer fix(core): flatten generations for LangChainTracer Dec 19, 2025
@github-actions github-actions bot added fix For PRs that implement a fix and removed breaking Breaking changes labels Dec 19, 2025
@mdrxy mdrxy changed the title fix(core): flatten generations for LangChainTracer fix(core): flatten generations for LangChainTracer Dec 19, 2025
@github-actions github-actions bot added fix For PRs that implement a fix and removed fix For PRs that implement a fix labels Dec 19, 2025
mdrxy
mdrxy previously approved these changes Dec 19, 2025
@github-actions github-actions bot added fix For PRs that implement a fix and removed fix For PRs that implement a fix labels Dec 19, 2025
@mdrxy mdrxy changed the title fix(core): flatten generations for LangChainTracer fix(core)!: flatten generations for LangChainTracer Dec 19, 2025
@github-actions github-actions bot added fix For PRs that implement a fix breaking Breaking changes and removed fix For PRs that implement a fix labels Dec 19, 2025
mdrxy added a commit that referenced this pull request Dec 19, 2025
…4414)

Adds `usage_metadata` (token counts, etc.) to the run metadata in
`LangChainTracer`.

When an LLM run ends, usage metadata is extracted from all generations
and aggregated using the existing `add_usage` helper, then stored in
`run.extra["metadata"]["usage_metadata"]`.

The original data in outputs remains unchanged.

Also, see #34415

---------

Co-authored-by: Mason Daugherty <[email protected]>
Co-authored-by: Mason Daugherty <[email protected]>
Base automatically changed from hunter/usage-metadata-extras to master December 19, 2025 18:59
@mdrxy mdrxy dismissed their stale review December 19, 2025 18:59

The base branch was changed.

@github-actions github-actions bot added breaking Breaking changes fix For PRs that implement a fix and removed fix For PRs that implement a fix breaking Breaking changes labels Dec 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaking changes core `langchain-core` package issues & PRs fix For PRs that implement a fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants