Skip to content

feat(examples): make LLM and vision model names configurable via env vars#231

Open
zhangzhenfei wants to merge 1 commit intoHKUDS:mainfrom
zhangzhenfei:feat/env-configurable-model-names
Open

feat(examples): make LLM and vision model names configurable via env vars#231
zhangzhenfei wants to merge 1 commit intoHKUDS:mainfrom
zhangzhenfei:feat/env-configurable-model-names

Conversation

@zhangzhenfei
Copy link
Copy Markdown

Summary

Currently, LLM and vision model names (gpt-4o-mini, gpt-4o) are hardcoded in example scripts. Users who rely on OpenAI-compatible providers (e.g. Azure, local inference servers, third-party proxies) must edit the Python source to change models — while EMBEDDING_MODEL and EMBEDDING_DIM are already read from environment variables.

This PR brings LLM/vision model configuration in line with the existing embedding pattern.

Changes

  • example_knowledge_qa.py: read LLM_MODEL, VISION_MODEL, EMBEDDING_MODEL, EMBEDDING_DIM from env (defaults preserved: gpt-4o-mini, gpt-4o, text-embedding-3-large, 3072)
  • examples/raganything_example.py: read LLM_MODEL, VISION_MODEL from env (same defaults)
  • env.example: document the new VISION_MODEL variable alongside LLM_MODEL

Usage after this change

Users can now switch models without touching code:

# .env
LLM_MODEL=kimi-k2.5
VISION_MODEL=kimi-k2.5
EMBEDDING_MODEL=text-embedding-3-large
EMBEDDING_DIM=3072

Test plan

  • Verify defaults are unchanged (existing OpenAI users unaffected)
  • Verify model names are picked up correctly when env vars are set
  • examples/lmstudio_integration_example.py already uses this pattern — confirm consistency

@LarFii
Copy link
Copy Markdown
Collaborator

LarFii commented Mar 24, 2026

Thanks for your contribution!

I did find one P1 blocker.

  1. P1: example_knowledge_qa.py's --query flow is broken and will raise at runtime.
    The new script builds a RAGAnything instance in initialize_rag() (PR file example_knowledge_qa.py, lines 99-116) and then calls rag.aquery(...) in the --query path (lines 135-142 and 187-192). But upstream aquery() explicitly raises when self.lightrag is None (raganything/query.py, lines 119-122). This script never initializes or reloads LightRAG before querying, so the advertised workflow (--process first, then --query) still fails on a fresh process. In practice, users following the example will hit a ValueError instead of getting an answer. This should be fixed before merge, either by ensuring the query path initializes/restores LightRAG from WORKING_DIR, or by using an API that performs that initialization before aquery().

- Read LLM_MODEL, VISION_MODEL, EMBEDDING_MODEL, EMBEDDING_DIM, PARSER
  from environment variables in example_knowledge_qa.py and
  examples/raganything_example.py (defaults preserved)
- Document new variables in env.example
- Fix --query path raising ValueError: call _ensure_lightrag_initialized()
  in initialize_rag() so LightRAG is loaded from WORKING_DIR before aquery()
@zhangzhenfei zhangzhenfei force-pushed the feat/env-configurable-model-names branch from 4a5a5a4 to fd7bec9 Compare March 24, 2026 07:38
@zhangzhenfei
Copy link
Copy Markdown
Author

Thanks for the review! The P1 blocker has been fixed — initialize_rag() now calls await rag._ensure_lightrag_initialized() before returning, so LightRAG is loaded from WORKING_DIR on every code path (including --query). The three commits have also been squashed into one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants