fix: anchor tool schema messages to English#302
fix: anchor tool schema messages to English#302mason5052 wants to merge 2 commits intovxcontrol:mainfrom
Conversation
## Summary Hard-codes the user-facing description text in the tool JSON schema to English so the LLM never sees the legacy "in user's language only" hint. This stops Russian/mixed-language `message` and `result` fields from leaking into traces and the UI when a flow is created in English. ## Problem backend/pkg/tools/args.go declared 29 jsonschema_description tags that told the model to answer "in user's language only". Because the schema is reflected once at startup and shared across flows, a flow in English would still receive the Russian-leaning hint baked into the tool definitions, producing the mixed-language output reported in issue vxcontrol#285. ## Solution Replace every `in user's language only` and the single `in the user's language only` occurrence with `in English` in the struct tags. No runtime locale logic, no schema re-shape, no API/DB change. Three remaining `in the user's language` (without "only") strings on different fields are intentionally left alone to keep this fix narrow. ## User Impact - New flows: tool-call message / result / clarification fields render in English regardless of UI locale. - No migration, config, or restart steps for users. - Flow.language is unchanged; if multilingual support is added later, this is a one-line edit per field. ## Test Plan - New test TestToolSchemasDoNotMentionUsersLanguageOnly sweeps every tool returned by GetRegistryDefinitions() and fails if the banned substring re-appears in any schema. - New test TestUserFacingMessageDescriptionsAreEnglishAnchored asserts the eight user-facing message/result properties carry the "in English" anchor, going through the same reflector path the LLM sees. - go test ./pkg/tools/... passes locally. - rg "user's language only" backend/pkg/tools/args.go returns no matches. Closes vxcontrol#285 Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR removes locale-ambiguous “in user's language only” hints from tool argument JSON schema descriptions and replaces them with an explicit “in English” anchor, preventing mixed-language tool-call message/result outputs (per #285).
Changes:
- Updated
jsonschema_descriptionstruct tags inbackend/pkg/tools/args.goto replace “in user's language only” with “in English” for user-facingmessage/resultfields. - Added regression tests to ensure tool schemas never reintroduce the banned substring and that key
message/resultdescriptions explicitly include “in English”.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| backend/pkg/tools/args.go | Re-anchors user-facing tool schema descriptions from “user’s language only” to “in English”. |
| backend/pkg/tools/args_test.go | Adds regression tests to prevent reintroduction of locale-ambiguous schema text and assert English anchoring for key fields. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The locale-ambiguous wording lived in three Search*Action.Message
descriptions ('SearchGuideAction', 'SearchAnswerAction',
'SearchCodeAction'). Replace 'in the user's language' with 'in English'
so the schema agrees with the system prompt, which already renders
{{.Lang}} as English under vxcontrol#216's English-only policy.
Strengthen TestToolSchemasDoNotInstructUsersLanguage so it does not
only reject the older exact substring 'user's language only'. It now
rejects 'user's language' outright, which also catches variants like
'the user's language' and prevents future schema edits from quietly
reintroducing the regression.
Signed-off-by: mason5052 <ehehwnwjs5052@gmail.com>
|
Pushed a follow-up (7437520) that broadens the regression guard. Previously the test only checked for the literal substring Renamed the test to |
|
PR body refreshed to reflect the follow-up commit (7437520). The earlier body said three
|
Summary
Hard-codes the user-facing description text in the tool JSON schema to English so the LLM never sees the legacy "in user's language only" hint. This stops Russian/mixed-language
messageandresultfields from leaking into traces and the UI when a flow is created in English.Problem
backend/pkg/tools/args.godeclaredjsonschema_descriptiontags that told the model to answer "in user's language only" / "in the user's language only" / "in the user's language". Because the schema is reflected once at startup and shared across flows, a flow in English would still receive the Russian-leaning hint baked into the tool definitions, producing the mixed-language tool-call output reported in #285.Solution
Replace every
in user's language only,in the user's language only, andin the user's languageoccurrence within Englishin the struct tags acrossbackend/pkg/tools/args.go. The follow-up commit also removed the remainingin the user's language(without "only") schema hints on the search actionMessagefields, so no schema description still instructs the model to answer in the user's language. No runtime locale logic, no schema re-shape, no API/DB/UI change.User Impact
message/result/ clarification fields render in English regardless of UI locale.Flow.languageis unchanged; if multilingual support is added later, this is a one-line edit per field.Test Plan
TestToolSchemasDoNotInstructUsersLanguagesweeps every tool returned byGetRegistryDefinitions()and fails if any of the broader forbidden patterns (user's language,the user's language) reappear in any tool schema, not just the old exactuser's language onlyphrase.TestUserFacingMessageDescriptionsAreEnglishAnchoredasserts the eight user-facingmessage/resultproperties carry the "in English" anchor, going through the samereflector.Reflectpath the LLM sees.go test ./pkg/tools/...passes locally.rg "user's language|the user's language" backend/pkg/tools/args.goreturns no matches (verified).Closes #285