Gemma 4 E2B emits parenthetical "word(suffix)" continuations under lower-temperature sampling

[UncreatedLabs]

Observation

Gemma 4 E2B (instruction-tuned) appears to have a learned "word(suffix)" parenthetical continuation style — the model frequently emits constructions like offline(text) / download(option) / feature(name), where a noun is immediately followed by a parenthesized clarifier word/phrase, in contexts where this format is not stylistically warranted.

Counter-intuitively, the pattern is more frequent at lower temperatures, suggesting it's a high-probability mode of the conditional distribution rather than a noise artifact. At Google's recommended T=1.0, topP=0.95, topK=64, the pattern is occasional. At T=0.4, it dominates body-text generation in our eval.

Why this matters

The expectation around lower temperature is "tighter, more deterministic, more on-task." For Gemma 4 E2B, lower temperature instead amplifies a specific learned-style attractor. This blocks one of the standard mitigations for a separate Gemma 4 decoding pathology — token repetition / corruption in tool arguments — that we and others (#622, vllm-project/vllm#40080) have documented.

Concretely: we'd lower temperature for tool-arg segments to reduce the mayaya/dzogogchen/202222-family corruption. But lowering it surfaces the parenthetical pattern in body text. We're stuck at T=1.0.

Reproduction

• Model: Gemma 4 E2B-it
• Runtime: LiteRT-LM 0.10.2 (also reproducible in vLLM per separate testing)
• Device: Pixel 10a, Mali-G715 GPU
• Sampler: topK=64, topP=0.95, T=0.4 (degraded), T=1.0 (acceptable but still occasional)
• Prompt class: general chat / explanatory turns. Most reproducible when the user asks "what is X" for technical X.

Related observations elsewhere

Closest published thing is #581 (degenerate embeddings producing fixed attractor states for rare-script tokens), and arXiv 2509.09715 on "symbolic triggers" of hallucination in Gemma. Neither addresses the parenthetical-suffix pattern specifically.

Ask

1. Acknowledgement that this is a known mode of Gemma 4 E2B's training distribution.
2. Guidance on whether instruction-tuned fine-tunes or future Gemma 4 revisions have addressed it.
3. Any data-side analysis that would help model consumers steer around it (preferred sampling profiles, system-prompt phrasing, etc.).

Happy to provide reproduction prompts + sample outputs.

References

• Field report: Gemma 4 E2B + LiteRT-LM 0.10.x on Mali-G715 — 10 findings from sustained Android testing google-ai-edge/LiteRT-LM#2202 — full field report on Gemma 4 E2B on-device behaviors
• Feature request: repetitionPenalty on SamplerConfig + per-segment sampler override - reproducible Gemma 4 tool-arg corruption catalog inside google-ai-edge/LiteRT-LM#2249 — feature request for repetitionPenalty / per-segment sampler
• [BUG] Gemma 4 token repetition collapse during long generation — affects both 31b Dense and 26b MoE #622 — token repetition collapse on larger variants
• Gemma3 4B: Degenerate embeddings for cuneiform/ancient script tokens cause deterministic hallucination #581 — cuneiform attractor state in Gemma 3

[sonali-kumari1]

Hi @UncreatedLabs -

Thanks for the detailed report. Could you please share a minimal reproducible example along with the prompts, any settings you have been using, and a few sample outputs across different temperatures?

Also, since you mentioned this issue is reproducible in both LiteRT-LM and vLLM could you clarify whether you’re seeing the same parenthetical continuation behavior consistently across both runtimes? Thanks!

[UncreatedLabs]

@sonali-kumari1 my pleasure, and thanks for commenting. We reproduced this and the data is below, but we owe everyone a correction first. The behaviour is GPU only, and it is not temperature driven. On the CPU backend we ran 34 generations total and they all came out clean. On the GPU backend, with the same prompts and the same sampler, the model deterministically corrupted specific tokens at every temperature we tried.

So this really looks like GPU numerical-precision corruption, the same family as the tool-argument corruption in HF discussion #10, rather than anything the model learned.

Setup

• Model: litert-community/gemma-4-E2B-it.litertlm
• Runtime: LiteRT-LM on a Pixel 10a (Tensor G4, Mali-G715, Android)
• Sampler: topK=64, topP=0.95, temperature swept across 0.4, 0.7, 1.0, 3 reps each
• Backends: CPU vs GPU, identical inputs
• Prompts: plain "what is X" explanatory questions, no tools (e.g. "What is a feature flag?", "What is dark mode?", "What is offline mode?", "What is a download manager?", "Explain what airplane mode does on a phone.")

"What is a feature flag?"

• CPU (T=0.4): A feature flag is a software development technique that allows you to turn specific features on or off in a live application without deploying new code. This lets developers control the release of new features gradually and manage risk.
• CPU (T=0.7): A feature flag is a software technique that allows developers to turn new features on or off for different users or environments without deploying new code. This lets companies test features safely and release them gradually.
• CPU (T=1.0): n/a (run cut off)
• GPU (T=0.4 / 0.7 / 1.0, identical): A feature flag is a software开关 that allows developers to turn specific features on or off in a live application without deploying new code. This lets teams test new features with real users or roll back problematic features quickly.
  • 开关 is Chinese for "switch". CPU never does this.

"What is dark mode?"

• CPU (T=0.4): Dark mode is a display setting where the background of a screen is dark, and the text and other foreground elements are light. It is designed to reduce eye strain, especially in low-light environments.
• CPU (T=0.7): Dark mode is a visual setting that inverts the colors of an interfaceand the text light. This is often done to reduce eye strain, especially in low-lightenvironments, and can also save battery life on OLED screens.
• CPU (T=1.0): n/a (run cut off)
• GPU (T=0.4 / 0.7 / 1.0, identical): Dark mode is a display setting that inverts the colors of a screen, making the background dark and the text light. It is designed to ลด eye strain, especially in low-light environments.
  • ลด is Thai for "reduce". CPU says "reduce".

Two things we want to be upfront about.

First, we are well past the build the original report came from. Those first examples were captured back in April on an older build with an earlier .litertlm snapshot, and we have since moved our on-device LLM off Gemma 4 E2B entirely. So the numbers above are a fresh rtert-community build, not the original setup. On these prompts the corruption showedup as foreign-script token substitution (开关, ลด) instead of the exact parenthetical and tool-argument manglings from the original report, and we could not reproduce those specific strings on the current binary. Same GPU-only root cause, different tokens.

Second, on the cross-runtime question: we cannot back up the idea that this reproduces in vLLM as a model behaviour. On our stack it follows the GPU (Mali) path specifically. CPU stays clean on identical inputs and temperature does not change anything, which points at runtime and hardware numerical precision rather than the model itself. We would treat our earlier "reproducible across runtimes" wording as something to walk back.