Skip to content

Stream llama.cpp model download progress - #27

Merged
jbouder merged 1 commit into
mainfrom
llamacpp-fix
May 1, 2026
Merged

jbouder merged 1 commit into
mainfrom
llamacpp-fix

Conversation

@jbouder

@jbouder jbouder commented May 1, 2026

Copy link
Copy Markdown
Contributor

Summary

This changes the llamacpp capability so model downloads no longer happen silently inside llama-server startup.

Instead of launching bin/llama-server -hf ... directly from Pixi, serve now routes through a new Python launcher:

  • pre-checks the requested Hugging Face GGUF repo and model selector
  • selects matching model .gguf files plus any mmproj GGUFs
  • runs hf download --dry-run to show cache/download intent
  • streams hf download output to the terminal before server startup
  • then starts llama-server -hf ... with the same CPU/GPU flags as before

The launcher keeps HF_HOME=models, so cached model files remain isolated to the capability and subsequent runs reuse the local cache.

@jbouder
jbouder merged commit 1e117dc into main May 1, 2026
1 check passed
@jbouder
jbouder deleted the llamacpp-fix branch May 1, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant