feat(models): per-provider model catalogs and model-derived context budget - #3087
feat(models): per-provider model catalogs and model-derived context budget#3087chimyves wants to merge 1 commit into
Conversation
|
Thanks for this — the direction is great. Per-provider catalogs plus a model-derived context budget is exactly the right foundation, and the catalog → seed → presets fallback keeps existing setups working. A few things before this can be merged:
Overall really nice work — this also unblocks a context-budget usage indicator and one-click compact/clear, which is why the accurate per-model |
Fixes #3085
What
models/model_catalog.py: per-provider model catalog persisted underprovider_model_catalogin config.json, keyed by provider id ("zhipu","custom:<id>"). Entries carry capability tags plus optionalcontext_window/max_output_tokens:textmarks a conversational model: text-tagged entries are the main-model (chat) candidates and what the session switcher offers.vision/video/image/embedding/asr/ttsroute entries into the matching capability cards. A model may hold several tags (text + vision for a VL model).custom_providers[].modelstays as the default model).POST /api/modelsgains asave_catalogaction (full-list replace with validation) andGET /api/modelsattaches each provider'scatalogplus aseed: the vendor's preset models pre-typed with their real capabilities and researched specs (context window / max output for ~50 built-in models, from official docs; native-multimodal models the preset lists lagged on — Kimi K3, MiniMax M3/M2.7, ERNIE 5.0 — get their vision tag). The vendor and custom-provider modals seed the editable model table from catalog → seed → presets._session_model_catalog) offers the provider's text-tagged models while keeping the "active model stays visible" behavior.agent_max_context_tokensis removed.Agent._get_model_context_window()prefers the catalog'scontext_window(falling back to the existing name heuristics) and_get_output_reserve_tokens()uses the catalog'smax_output_tokenswhen set, so the trimming budget follows the effective model automatically — a session that switched models uses that model's window, message channels the main model's. Sub-agents keep inheriting an explicit cap (never above the input ceiling).max_output_tokensis passed through to the LLM request only when configured, so gateways that reject an explicitmax_tokensare unaffected.Why
One-model-per-provider makes multi-model setups impossible to maintain, and the global context budget ignores the active model (small-window models 400 with an oversized budget; large windows get silently capped). This is also the prerequisite for model routing/fallback (#2747) and gives the session model switcher (#2995) a real list to switch between.
Notes
agent_max_context_tokensvalue is simply ignored.