feat: add eval feedback helper for onResult callbacks and VoltOps fee…

[omeraplak]

…dback

PR Checklist

Please check if your PR fulfills the following requirements:

• The commit message follows our guidelines: https://voltagent.dev/docs/community/contributing/#commit-convention

Bugs / Features

• Related issue(s) linked
• Tests for the changes have been added
• Docs have been added / updated
• Changesets have been added https://voltagent.dev/docs/community/contributing/#creating-a-changeset

What is the current behavior?

What is the new behavior?

fixes (issue)

Notes for reviewers

---

Summary by cubic

Adds a feedback helper to eval scorer onResult callbacks and VoltOps feedback API support, so scorers can save feedback to VoltOps with one call. Updates examples and exports the new types.

• New Features

  • onResult now receives { result, feedback } and supports feedback.save({ key, score, value, ... }).
  • VoltOps client adds createFeedback and new types (VoltOpsFeedback, VoltOpsFeedbackCreateInput).
  • Agent exposes a validated VoltOps client to evals; feedback.save no-ops if VoltOps keys are missing.
  • Examples add helpfulness judge and show saving model feedback to VoltOps.

• Migration

  • Update onResult signatures to accept a single args object and call args.feedback.save(...).
  • If you reference types, import AgentEvalResultCallbackArgs, AgentEvalFeedbackHelper, and AgentEvalFeedbackSaveInput from @voltagent/core.

^{Written for commit 6b757ab. Summary will update on new commits.}

Summary by CodeRabbit

• New Features

  • Added evaluation feedback persistence through VoltOps integration, enabling evaluation results to be automatically saved and stored.
  • Added new example implementations demonstrating feedback-based evaluation workflows, including task scorers and LLM-based satisfaction judges.

• Documentation

  • Added documentation for native feedback helper integration patterns and usage examples.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[changeset-bot]

🦋 Changeset detected

Latest commit: 6b757ab

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@voltagent/core	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds VoltOps feedback support to the agent evaluation system. Introduces new types for feedback creation and helper objects, adds getVoltOpsClient() and createFeedback() methods, integrates feedback persistence into eval result callbacks, and provides documentation and examples for the feedback workflow.

Changes

Cohort / File(s)	Summary
Type Definitions & VoltOps API packages/core/src/voltops/types.ts, packages/core/src/voltops/index.ts	Introduces VoltOpsFeedbackCreateInput and VoltOpsFeedback types to define feedback entity schema with snake_case fields (trace_id, feedback_source, feedback_source_type, etc.) and optional properties.
Agent Evaluation Feedback Types packages/core/src/agent/types.ts, packages/core/src/index.ts	Adds public types: AgentEvalFeedbackSaveInput, AgentEvalFeedbackHelper, and AgentEvalResultCallbackArgs; updates AgentEvalScorerConfig.onResult signature to receive the new callback args structure.
VoltOps Client Method packages/core/src/voltops/client.ts	Implements createFeedback(input: VoltOpsFeedbackCreateInput) method to POST feedback to /api/public/feedback endpoint and return created feedback object.
Agent Evaluation Feedback Integration packages/core/src/agent/eval.ts, packages/core/src/agent/agent.ts	Adds getVoltOpsClient() method to evaluation host; introduces createEvalFeedbackHelper() and resolveEvalFeedbackClient() functions; wraps eval results with feedback helper in invokeEvalResultCallback, enabling persistence via feedback.save() with key/traceId validation and debug logging.
Documentation & Examples .changeset/native-feedback-helper.md, examples/with-live-evals/src/index.ts	Documents native feedback helper usage patterns with task-scorer and LLM-judge examples; adds helpfulnessJudgeAgent, customScorer, and demo agents (singleEvalAgent, scorerFeedbackAgent) showcasing feedback integration via onResult callbacks.

Sequence Diagram

sequenceDiagram
    participant Scorer as Eval Scorer
    participant EvalHost as Evaluation Host
    participant FeedbackHelper as Feedback Helper
    participant VoltOpsClient as VoltOps Client
    participant API as /api/public/feedback

    Scorer->>EvalHost: produce evaluation result
    EvalHost->>FeedbackHelper: createEvalFeedbackHelper(host, result)
    EvalHost->>EvalHost: invokeEvalResultCallback(AgentEvalResultCallbackArgs)
    
    Note over EvalHost: onResult callback invoked<br/>with result + feedback helper
    
    EvalHost->>FeedbackHelper: user calls feedback.save(input)
    FeedbackHelper->>FeedbackHelper: validate key & traceId
    FeedbackHelper->>EvalHost: resolveEvalFeedbackClient(host)
    EvalHost->>VoltOpsClient: getVoltOpsClient()
    VoltOpsClient-->>FeedbackHelper: return validated client
    
    FeedbackHelper->>VoltOpsClient: createFeedback(payload)
    VoltOpsClient->>API: POST with traceId, key, score, etc.
    API-->>VoltOpsClient: return VoltOpsFeedback
    VoltOpsClient-->>FeedbackHelper: return persisted feedback
    FeedbackHelper-->>EvalHost: Promise<VoltOpsFeedback | null>

Loading

Estimated Code Review Effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly Related PRs

• ci(changesets): version packages #941: Adds VoltOps feedback support in core with types, APIs, and evaluation feedback hooks that align with this PR's integration patterns.
• feat: add feedback #940: Modifies codebase to add VoltOps feedback methods and related types with evaluation/agent feedback wiring similar to this change.

Poem

🐰 A feedback journey, swift and bright,
Scorers hop with eval delight,
Results saved through VoltOps' door,
Each carrot score, forevermore! 🥕✨

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description includes required sections but most are incomplete: current/new behavior sections are empty, the 'fixes' reference is a placeholder, and all PR checklist items are unchecked without context.	Complete the 'What is the current behavior?' and 'What is the new behavior?' sections, remove or properly fill the 'fixes (issue)' placeholder, check applicable checklist items, and fill 'Notes for reviewers' if needed. The auto-generated summary is helpful but doesn't replace these required sections.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes the main feature: adding an eval feedback helper for onResult callbacks and VoltOps feedback support, which aligns with the primary changes across the codebase.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

🧹 Recent nitpick comments

packages/core/src/agent/agent.ts (1)

2689-2698: Consider extracting the common VoltOps client resolution logic.

The implementation is correct and follows the established pattern. However, this logic is nearly identical to getFeedbackClient() at lines 2749-2758. Consider extracting the shared client resolution into a single private helper to reduce duplication.

♻️ Optional: Extract shared client resolution

+ private getValidatedVoltOpsClient(): VoltOpsClient | undefined {
+   const client = this.voltOpsClient || AgentRegistry.getInstance().getGlobalVoltOpsClient();
+   if (!client || typeof client.hasValidKeys !== "function") {
+     return undefined;
+   }
+   if (!client.hasValidKeys()) {
+     return undefined;
+   }
+   return client;
+ }

  private createEvalHost(): AgentEvalHost {
    return {
      // ...
-     getVoltOpsClient: () => {
-       const client = this.voltOpsClient || AgentRegistry.getInstance().getGlobalVoltOpsClient();
-       if (!client || typeof client.hasValidKeys !== "function") {
-         return undefined;
-       }
-       if (!client.hasValidKeys()) {
-         return undefined;
-       }
-       return client;
-     },
+     getVoltOpsClient: () => this.getValidatedVoltOpsClient(),
    };
  }

  private getFeedbackClient(): VoltOpsClient | undefined {
-   const voltOpsClient =
-     this.voltOpsClient || AgentRegistry.getInstance().getGlobalVoltOpsClient();
-   if (!voltOpsClient || typeof voltOpsClient.hasValidKeys !== "function") {
-     return undefined;
-   }
-   if (!voltOpsClient.hasValidKeys()) {
-     return undefined;
-   }
-   return voltOpsClient;
+   return this.getValidatedVoltOpsClient();
  }

packages/core/src/voltops/client.ts (1)

287-322: Consider adding response validation for consistency.

The implementation correctly follows the established pattern for payload construction. However, unlike createFeedbackToken which validates the response fields (lines 275-277), createFeedback directly returns the API response without validation.

If the API could return malformed data, consider adding defensive validation similar to createFeedbackToken:

♻️ Optional: Add response validation

  public async createFeedback(input: VoltOpsFeedbackCreateInput): Promise<VoltOpsFeedback> {
    const payload: Record<string, unknown> = {
      trace_id: input.traceId,
      key: input.key,
    };

    // ... conditional field additions ...

-   return await this.request<VoltOpsFeedback>("POST", "/api/public/feedback", payload);
+   const response = await this.request<VoltOpsFeedback>("POST", "/api/public/feedback", payload);
+
+   if (!response?.id || !response?.traceId || !response?.key) {
+     throw new Error("Failed to create feedback via VoltOps: invalid response");
+   }
+
+   return response;
  }

---

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a803d31 and 6b757ab.

📒 Files selected for processing (9)

• .changeset/native-feedback-helper.md
• examples/with-live-evals/src/index.ts
• packages/core/src/agent/agent.ts
• packages/core/src/agent/eval.ts
• packages/core/src/agent/types.ts
• packages/core/src/index.ts
• packages/core/src/voltops/client.ts
• packages/core/src/voltops/index.ts
• packages/core/src/voltops/types.ts

🧰 Additional context used

📓 Path-based instructions (1)

**/*.ts

📄 CodeRabbit inference engine (AGENTS.md)

**/*.ts: Maintain type safety in TypeScript-first codebase
Never use JSON.stringify; use the safeStringify function instead, imported from @voltagent/internal

Files:

• packages/core/src/index.ts
• packages/core/src/voltops/index.ts
• packages/core/src/voltops/types.ts
• examples/with-live-evals/src/index.ts
• packages/core/src/agent/eval.ts
• packages/core/src/agent/types.ts
• packages/core/src/voltops/client.ts
• packages/core/src/agent/agent.ts

🧠 Learnings (2)

📚 Learning: 2026-01-07T05:09:23.227Z

Learnt from: CR
Repo: VoltAgent/voltagent PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-07T05:09:23.227Z
Learning: Always check existing patterns before implementing new features in VoltAgent

Applied to files:

• packages/core/src/agent/eval.ts
• packages/core/src/agent/agent.ts

📚 Learning: 2026-01-07T05:09:23.227Z

Learnt from: CR
Repo: VoltAgent/voltagent PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-07T05:09:23.227Z
Learning: Use the established registry patterns for agent and tool management in VoltAgent

Applied to files:

• packages/core/src/agent/agent.ts

🧬 Code graph analysis (3)

packages/core/src/agent/eval.ts (3)

packages/core/src/voltops/types.ts (1)

• VoltOpsClient (960-998)

packages/core/src/agent/types.ts (4)

• AgentEvalResultCallbackArgs (573-576)
• AgentEvalResult (553-563)
• AgentEvalFeedbackHelper (569-571)
• AgentEvalFeedbackSaveInput (565-567)

packages/core/src/index.ts (4)

• AgentEvalResultCallbackArgs (177-177)
• AgentEvalResult (176-176)
• AgentEvalFeedbackHelper (178-178)
• AgentEvalFeedbackSaveInput (179-179)

packages/core/src/agent/types.ts (3)

packages/core/src/index.ts (4)

• AgentEvalFeedbackSaveInput (179-179)
• AgentEvalFeedbackHelper (178-178)
• AgentEvalResultCallbackArgs (177-177)
• AgentEvalResult (176-176)

packages/core/src/voltops/index.ts (2)

• VoltOpsFeedbackCreateInput (40-40)
• VoltOpsFeedback (39-39)

packages/core/src/voltops/types.ts (2)

• VoltOpsFeedbackCreateInput (156-168)
• VoltOpsFeedback (170-185)

packages/core/src/agent/agent.ts (1)

packages/core/src/registries/agent-registry.ts (1)

• AgentRegistry (16-233)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)

• GitHub Check: cubic · AI code reviewer
• GitHub Check: Test core
• GitHub Check: Test libsql
• GitHub Check: Test supabase
• GitHub Check: Build (Node 24)
• GitHub Check: Build (Node 22)
• GitHub Check: Build (Node 20)
• GitHub Check: Cloudflare Pages

🔇 Additional comments (23)

packages/core/src/voltops/index.ts (1)

39-40: LGTM!

The new type exports VoltOpsFeedback and VoltOpsFeedbackCreateInput are properly grouped with related feedback types and follow the established export pattern.

packages/core/src/index.ts (1)

177-179: LGTM!

The new evaluation feedback types are properly grouped with related AgentEval* types and follow the established export conventions.

.changeset/native-feedback-helper.md (1)

1-107: LGTM!

The changeset documentation is well-structured with clear, practical examples demonstrating both simple rule-based scoring and LLM-based judge patterns. The examples effectively illustrate the new feedback.save() API.

packages/core/src/voltops/client.ts (1)

58-60: LGTM!

The new type imports are properly added and align with the new createFeedback method implementation.

packages/core/src/agent/types.ts (3)

33-41: Import additions look fine.
No concerns with the VoltOps type imports.

---

565-577: Types for feedback helper are well-scoped.
Clear separation between save input, helper, and callback args.

---

578-588: Updated onResult callback typing is clear.
Nice to surface both result and feedback in the callback args.

examples/with-live-evals/src/index.ts (8)

24-28: Helpfulness judge agent setup looks good.
Clean separation for the evaluation model.

---

70-76: Custom scorer stub is fine for the demo.
Straightforward and readable.

---

134-134: Judge call is correctly wired.
Schema-based object generation fits the scorer flow.

---

253-266: Feedback persistence in onResult is well-structured.
Good use of the helper with metadata attribution.

---

299-311: Single-eval agent configuration looks good.
Clear, minimal example.

---

313-339: Scorer-feedback agent wiring is solid.
Consistent with the support agent feedback flow.

---

342-346: Agent registry update is fine.
Nice to expose the new demo agents.

---

353-353: Demo invocation aligns with new single-eval flow.
No issues.

packages/core/src/voltops/types.ts (2)

156-185: Feedback types are comprehensive and consistent.
The create input vs stored shape is clear.

---

976-977: Client interface extension is straightforward.
Good to surface feedback creation in the public API.

packages/core/src/agent/eval.ts (6)

19-19: Import addition is fine.
No concerns.

---

23-29: Type imports align with new feedback flow.
Looks good.

---

255-262: Host interface update is sensible.
Optional VoltOps client getter keeps compatibility.

---

1099-1106: Callback payload wrapping is clean.
Great to standardize result + feedback for onResult.

---

1115-1155: Feedback helper implementation is solid.
Validation + safe fallback when client is unavailable is a good touch.

---

1157-1172: Client resolution guardrails look good.
Key validation hook is a nice safety net.

_{✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[cubic-dev-ai]

No issues found across 9 files

[cloudflare-workers-and-pages]

Deploying voltagent with    Cloudflare Pages

Latest commit:	6b757ab
Status:	✅  Deploy successful!
Preview URL:	https://bd5bf008.voltagent.pages.dev
Branch Preview URL:	https://feat-feedback-scorers.voltagent.pages.dev

View logs