coli run: SNAP env var never set for non-glm engines (olmoe confirmed) — always fails with 'started without a model'

[adirburke]

Summary

coli run (and any one-shot invocation of openai_server-adjacent gateway) never sets the SNAP environment variable for non-glm engine families. The engines themselves (olmoe.c confirmed, likely inkling.c/kimi_k3.c/deepseek_v4.c too since they share the same env_for_engine() launcher path) read SNAP to locate the model directory, so coli run always fails with "started without a model" even when the exact same --model directory passes coli doctor cleanly.

Root cause

In c/coli:

• env_for_engine(a, arch) (around line 426) only special-cases arch == "glm", delegating to env_for(a), which is the only function that sets SNAP (e = dict(os.environ, SNAP=a.model), around line 679).
• For every other arch, env_for_engine builds its own env dict from os.environ.copy() and never sets SNAP.
• cmd_run's olmoe branch (around line 1300) calls the engine directly:

  e = env_for_engine(a, arch)
  result = subprocess.run([engine, str(cap_for_launch(a.cap, e, 16)), "8"],
                           input=prompt + "\n", text=True, env=e, check=False)

  with no SNAP in e and no model path passed as an argument either.
• olmoe.c reads the model path exclusively via getenv("SNAP") (line ~1585).

Contrast with coli chat/coli serve for the same non-glm archs: those spawn openai_server.py as a subprocess, and openai_server.py sets SNAP itself when it in turn launches the engine (child_env = dict(env or os.environ, SNAP=str(model), SERVE="1", ...), around line 2938). That's why chat/serve work while run does not.

Repro

cd c
make olmoe
python3 tools/convert_olmoe_merged.py --repo allenai/OLMoE-1B-7B-0125-Instruct --out ./olmoe_merged
python3 ./coli doctor --model ./olmoe_merged     # -> result ok
python3 ./coli run --model ./olmoe_merged "hi"   # -> fails

Output of the failing run:

colibri: this is the OLMoE engine, and it was started without a model.
The engine is not the program you run directly -- the launcher is:
    ./coli chat  --model <model directory>    interactive chat
    ...

immediately followed by the run banner (OLMoE · 7B · 7.4 GB on disk / run) and a non-zero exit — the launcher believes it found and validated the model (via need_model/resolve_model), then hands the engine an environment missing SNAP.

Manually adding SNAP confirms the engine and model are otherwise fine:

SNAP="$(pwd)/olmoe_merged" CHAT=1 MAX_NEW=100 ./olmoe 16 8 <<< "Explain what a mixture-of-experts model is."
# -> loads, generates a coherent answer in ~9s

Environment

• macOS 26.6.2, Apple M3 Max, colibri v1.11.0, built from source (make olmoe, no METAL — this family has no Metal path).
• Confirmed on the olmoe arch; worth checking whether inkling/kimi_k3/deepseek_v4's cmd_run branches have the same gap, since they share env_for_engine().

Suggested fix

Either have env_for_engine() set SNAP=a.model unconditionally (mirroring what env_for() does for glm, and what openai_server.py already does when it spawns these same engines), or have each non-glm branch of cmd_run pass SNAP explicitly before invoking the engine directly.

[JustVugg]

Your diagnosis is exactly right, down to the two call sites and the reason chat and serve worked while run did not. It is also already fixed on dev, and the fix is the first of the two you proposed.

Commit ea846033 (14 September, "coli run: hand every engine its model directory through SNAP") puts it in env_for_engine() itself:

# SNAP is how every engine but GLM learns its model directory. env_for()
# sets it for GLM and openai_server.py sets it for the gateway, so `coli
# chat` and `coli serve` worked while `coli run` handed the sister engines
# an environment without it: OLMoE exited with "started without a model"
# (#1501). Set it here, once, for all of them.
if getattr(a, "model", None):
    env.setdefault("SNAP", os.path.abspath(a.model))

That closes #1501, which is the same bug reported on Linux. Yours is the first macOS confirmation of it, which is worth having: it rules out anything platform-specific in the path.

On your open question about the sibling engines: you were right, and the same commit covers them. The line sits in env_for_engine() above the per-arch branches, and every non-GLM arch reaches it (cmd_run at three call sites, plus cmd_chat, the tuner and the serve path), so inkling, kimi_k3, deepseek_v4 and the rest get it from the one change rather than needing a branch each.

What it means for you right now: the commit is dated after the v1.11.0 tag and is not on main, so a source build of v1.11.0 does not have it. Either build from dev, or keep the SNAP=... you already found, which is precisely what the fix does for you.

git clone --branch dev --single-branch https://github.com/JustVugg/colibri.git

dev has moved a fair way since the tag, including a few OLMoE and macOS items, so it is worth a fresh build rather than a cherry-pick if you are going to test more of this on the M3.

Leaving this open until the next release carries the fix, so it stays findable for anyone else hitting it on v1.11.0.