Skip to content

Conversation

vblagoje
Copy link
Member

@vblagoje vblagoje commented Aug 22, 2025

Why:

Introduces support for Gemini "thinking/reasoning" .Models can now show their reasoning process, maintain thought context across multi-turn conversations, and use configurable thinking budgets.

What:

  • Added thinking_budget configuration via generation_kwargs with model-specific validation
  • Implemented parsing of thought parts into ReasoningContent objects
  • Extended streaming aggregation to handle reasoning deltas while reusing generic aggregator
  • Implemented fail-fast error handling for unsupported models
  • Added integration tests for thinking feature and error scenarios

How can it be used:

from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator
from haystack.dataclasses import ChatMessage

# Enable thinking with dynamic or explicit budget
generator = GoogleGenAIChatGenerator(
    model="gemini-2.5-pro",
    generation_kwargs={"thinking_budget": -1}  # -1 for dynamic, or specific token count
)

messages = [ChatMessage.from_user("Explain why the sky is blue")]
response = generator.run(messages=messages)

# Access reasoning content
message = response["replies"][0]
if message.reasonings:
    for reasoning in message.reasonings:
        print(f"Thought: {reasoning.reasoning_text}")

# Check thinking tokens used
print(f"Thinking tokens: {message.meta['usage'].get('thoughts_token_count', 0)}")

How did you test it:

  • Added integration tests for thinking feature with gemini-2.5-pro
  • Added fail-fast tests for unsupported models

Notes for the reviewer:

  • _aggregate_streaming_chunks_with_reasoning extends the generic aggregator (40 lines vs 78 if duplicated)
  • Model detection may need updates as new models are released

@github-actions github-actions bot added integration:google-genai type:documentation Improvements or additions to documentation labels Aug 22, 2025
@vblagoje vblagoje marked this pull request as ready for review August 22, 2025 12:35
@vblagoje vblagoje requested a review from a team as a code owner August 22, 2025 12:35
@vblagoje vblagoje requested review from julian-risch and anakin87 and removed request for a team August 22, 2025 12:35
@anakin87
Copy link
Member

My impression is that handling thinking + tool calls have some peculiarities to take into consideration
https://ai.google.dev/gemini-api/docs/function-calling?example=meeting#thinking
https://ai.google.dev/gemini-api/docs/thinking#signatures

@vblagoje
Copy link
Member Author

My impression is that handling thinking + tool calls have some peculiarities to take into consideration https://ai.google.dev/gemini-api/docs/function-calling?example=meeting#thinking https://ai.google.dev/gemini-api/docs/thinking#signatures

Ok let me check

@vblagoje vblagoje marked this pull request as draft August 22, 2025 12:44
@vblagoje vblagoje changed the title feat: Add Gemini "Thinking" Support to GoogleGenAIChatGenerator feat: Add Gemini "Thinking" support to GoogleGenAIChatGenerator Aug 22, 2025
@vblagoje vblagoje marked this pull request as ready for review August 25, 2025 12:49
@@ -797,6 +808,115 @@ def test_live_run_with_parallel_tools(self, tools):
# Check that the response mentions both temperature readings
assert "22" in message.text or "15" in message.text

Copy link
Member

@anakin87 anakin87 Aug 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add unit tests to cover the new methods and code paths introduced in this PR

Copy link
Member

@anakin87 anakin87 Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the CI, I see this coverage

Name                                                                                  Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------------------------------------------------------------
src/haystack_integrations/components/generators/google_genai/chat/chat_generator.py     338     53    156     32    80%   77-78, 99-106, 108->118, 110->118, 123->128, 136-137, 141-142, 145-150, 161-163, 184-188, 195-205, 228-229, 275->309, 278->309, 305->279, 542, 564->568, 611-616, 631, 670->676, 672->671, 677-681, 684->666, 692, 697, 737-739, 772-774, 850-851, 862, 866, 904-905, 959-960, 971, 975, 1013-1014

Could you please add some additional unit tests to cover what changed in this PR?
For example, _convert_message_to_google_genai_format...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice nice, on it! One sec 🙏

@vblagoje
Copy link
Member Author

@julian-risch @anakin87 is this one good to go? LMK

@vblagoje vblagoje requested a review from anakin87 August 27, 2025 13:25
Copy link
Member

@anakin87 anakin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok for me.
My impression is that @julian-risch wants to review as well.

@vblagoje
Copy link
Member Author

@julian-risch would you please take a look - I'd like to integrate this one and release a new version soon

@vblagoje vblagoje merged commit 417404b into main Aug 28, 2025
11 checks passed
@vblagoje vblagoje deleted the googlegenai_thinking branch August 28, 2025 13:10
@vblagoje
Copy link
Member Author

@julian-risch I think @anakin87 is a very good reviewer so I took the liberty to move this PR forward.

@vblagoje
Copy link
Member Author

Included in https://pypi.org/project/google-genai-haystack/2.1.2/ release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integration:google-genai type:documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Google GenAI - reasoning support
2 participants