Skip to content

Reasoning control for OpenRouter models; accept reasoning_effort - #153

Open
cboettig wants to merge 1 commit into
mainfrom
openrouter-reasoning-toggle
Open

cboettig wants to merge 1 commit into
mainfrom
openrouter-reasoning-toggle

Conversation

@cboettig

Copy link
Copy Markdown
Member

Problem

enable_thinking only reaches NRP models listed in thinking_models (as chat_template_kwargs). For OpenRouter ids the proxy drops it with an info log, so an app cannot turn reasoning down on an OpenRouter model.

This shows up in global-30x30. In session b9f2c027 (2026-10-16), MiMo V2.6 Flash took 188 s on the step after fit_sdm, producing 4,985 completion tokens. 4,759 of those were reasoning, before a single add_hex_tile_layer call. Other steps in the same session spent 1,059 of 1,258 and 1,058 of 1,321 tokens reasoning.

Change

  • Provider dialect. New provider setting thinking_dialect; config.json sets "openrouter" on the OpenRouter provider. For a model not in thinking_models, the proxy then sends OpenRouter's top-level reasoning object.
  • Effort levels. New request field reasoning_effort (none / minimal / low / medium / high, OpenAI-style). If both fields are sent, the level wins. This gives clients a level to expose, as harnesses usually do; the boolean alone can't express one.
  • Translation. reasoning_payload() does the per-backend mapping:
Request NRP thinking_models model OpenRouter Other
enable_thinking: false {key: false} reasoning: {enabled: false} nothing
enable_thinking: true {key: true} reasoning: {enabled: true} nothing
reasoning_effort: "none" {key: false} reasoning: {enabled: false} nothing
reasoning_effort: "low" etc. {key: true} (logged: no levels) reasoning: {effort: "low"} nothing
invalid level ignored (logged); boolean still applies same same
  • Logging. reasoning_effort is logged on the request record next to enable_thinking.
  • Docs and tests. README "Thinking mode" section updated, CHANGELOG entry added. New tests cover the mapping for NRP, OpenRouter and a provider with no dialect. python test_logging.py passes the 45 tests before its __main__ block, including the three new ones.

Not verified yet

I could not probe OpenRouter directly; the key lives only in the cluster secret. After deploy, check the effect on MiMo Flash by sending the same request with "enable_thinking": false and with "reasoning_effort": "low", and comparing tokens.completion_tokens_details.reasoning_tokens in the response log. For models whose reasoning is mandatory, OpenRouter may still reason. Following the thinking_models precedent (#105), any model that ignores the setting should be recorded once probed.

Follow-ups

  • geo-agent: expose an effort selector and per-model reasoning_effort defaults in config, sending the new field. To be filed as a separate geo-agent issue.
  • global-30x30: once this is deployed, set a lower reasoning default for MiMo Flash.

enable_thinking reached only NRP models listed in thinking_models and was
dropped for OpenRouter ids. Add a thinking_dialect provider setting
(openrouter -> top-level reasoning object), accept an OpenAI-style
reasoning_effort level, translate both per backend in reasoning_payload(),
and log reasoning_effort on the request record.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant