Skip to content

/api/ps and /api/tags report size and size_vram as a hardcoded 0 #11969

Description

@YourYoungerBrothersPug

LocalAI version:
v4.9.0 (f7ad3f7), image localai/localai:v4.9.0-gpu-nvidia-cuda-12

Environment, CPU architecture, OS, and Version:

x86_64, RTX 3060 12GB, Docker on WSL2

Describe the bug
GET /api/ps reports "size": 0 and "size_vram": 0 for every loaded model. The
values are literals rather than unpopulated fields:

https://github.com/mudler/LocalAI/blob/v4.9.0/core/http/endpoints/ollama/models.go#L106-L110

GET /api/tags has the same literal Size: 0 at line 40, and ExpiresAt on the ps
entry is synthesised as time.Now().Add(24 * time.Hour).

This is worse than omitting the fields. /api/ps is an Ollama-compatibility surface,
and Ollama reports real byte counts there, so a client written against Ollama reads
0 as authoritative and concludes the models cost nothing. Anything scheduling GPU
work from that will over-commit the card. An absent field or a null is safe, because
a consumer can distinguish "unknown" from "nothing"; a 0 cannot be distinguished.

To Reproduce

  1. Load any model (granite-4.1-8b, llama-cpp backend) so it is resident.
  2. curl localhost:8080/api/ps
  3. Compare against nvidia-smi.

Expected behavior
Either the real resident sizes, or the fields omitted/null when the backend cannot
report them. Not 0.

Logs

{"models":[{"name":"granite-4.1-8b:latest","size":0,"size_vram":0,
  "details":{"format":"gguf","family":"llama-cpp","parameter_size":"8B",
  "quantization_level":"Q4_K_M"}}]}

Card at the same moment: 10849 / 12288 MiB used, two models resident (second row
elided, also all-zero).

Additional context
I gave up on /api/ps for this engine entirely and treat the size as unknown unconditionally, because I can't tell its zeros from real ones.

Filed separately as a feature request: there is no endpoint that reports per-model memory at all.

I'm not in a position to take the PR, but I'm happy to test a fix against this setup.

Written by my beloved Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions