Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support logprobsResult and avgLogprobs Response from Google Generative AI #4969

Open
elikor-figma opened this issue Feb 24, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@elikor-figma
Copy link

Feature Description

The Google Generative AI inference API allows for a user to set the responseLogprobs field to true in their request to gemini-1.5-flash, along with the optional logprobs field to specify the number of candidate logprobs in each step. When responseLogprobs is set to true, the response then includes logprobsCandidates in the response (see in the response body schema here). It additionally appears like the avgLogprobs field is returned regardless of responseLogprobs being sent, and would additionally be useful to include. I would like these fields to be exposed in some way (perhaps through provider metadata) in the request and response of the AI SDK functions using the Google Generative AI provider.

Use Cases

Logprobs can be useful for filtering and selecting different token responses, providing a means of assigning confidence to different LLM outputs.

Additional context

No response

@elikor-figma elikor-figma added the enhancement New feature or request label Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant