Skip to content

Repository files navigation

Manifesto

Manifesto renders shareable Kubernetes manifests for LLM deployments.

It is built for the workflow where an engineer wants to describe a concrete vLLM deployment, render plain YAML, inspect or hand-edit it, and then apply exactly that artifact to a cluster.

Manifesto keeps intelligence in resolution, not in the artifact: model specs express intent, feature contracts supply required runtime details, and the result remains ordinary, minimal Kubernetes YAML.

The project is packaged as llm-manifesto, with manifesto as the Python package and CLI command name.

What It Does

Manifesto takes two YAML inputs:

  • a model spec: model image, topology, roles, parallelism, routing, vLLM args
  • a cluster profile: GPU shape, storage paths, fabric env, llm-d images

It emits raw Kubernetes manifests:

  • Deployment or LeaderWorkerSet model-server workloads, depending on node count
  • InferencePool and endpoint picker deployment
  • standalone Envoy routing by default, or optional Gateway API objects
  • per-pod monitoring sidecars
  • instance-scoped names, labels, selectors, and cache paths

The rendered YAML starts with provenance comments:

# Generated by:
#   uvx --from git+https://github.com/neuralmagic/llm-manifesto@<commit> manifesto render manifest \
#     <(cat <<'MANIFESTO_MODEL'
# <expanded model YAML>
# MANIFESTO_MODEL
#     ) \
#     --cluster <(cat <<'MANIFESTO_CLUSTER'
# <expanded cluster YAML>
# MANIFESTO_CLUSTER
#     )
# To set identity or placement, append --user USER and/or --namespace NAMESPACE.
# Source: https://github.com/neuralmagic/llm-manifesto/tree/<commit>
# Safe to edit before applying.
---
apiVersion: v1
kind: ServiceAccount
...

That header is intentional: rendered manifests should be easy to paste into a PR, send to another engineer, edit manually, and regenerate later.

License

Licensed under the Apache License, Version 2.0. See LICENSE.

Quick Start

Bootstrap namespace prerequisites declared by the cluster profile (such as a shared cache PVC):

manifesto render bootstrap --cluster clusters/example-gb200.yaml --namespace "$MANIFESTO_NAMESPACE"
manifesto deploy bootstrap --cluster clusters/example-gb200.yaml --namespace "$MANIFESTO_NAMESPACE"

Render a manifest:

uv run manifesto render manifest models/qwen/aggregated.yaml \
  --cluster clusters/example-gb200.yaml \
  --user "$USER"

Render to a file, edit it, diff it, then apply it:

manifesto render file models/deepseek-v4/1P-EP8-1D-EP8.yaml
$EDITOR /tmp/manifesto.yaml
manifesto file diff
manifesto file apply

Deploy directly:

manifesto deploy models/deepseek-v4/1P-EP8-1D-EP8.yaml
manifesto ready models/deepseek-v4/1P-EP8-1D-EP8.yaml

Stop the instance:

manifesto servers
manifesto stop                         # interactive picker
manifesto stop models/deepseek-v4/1P-EP8-1D-EP8.yaml
manifesto stop --instance "$USER-wide-ep-1p-ep8-1d-ep8"
manifesto stop models/deepseek-v4/1P-EP8-1D-EP8.yaml --now

Repository Layout

manifesto/                  Python renderer implementation
  cli.py                CLI entrypoints
  spec.py               model/deployment spec schema
  cluster.py            cluster profile schema
  instance.py           instance-scoped names, labels, selectors, paths
  resolve.py            concrete env, paths, ports, and vLLM args per role
  render/               Kubernetes object emitters

clusters/               cluster profiles
models/                 model deployment specs
tests/                  renderer and UX tests

User Configuration

Manifesto also reads a private user catalog from ~/.config/llm-manifesto (or $XDG_CONFIG_HOME/llm-manifesto). Keep local cluster and model configuration outside the repository with this layout:

~/.config/llm-manifesto/
  .env                         optional local defaults
  clusters/
    my-context.yaml
  models/
    model_provider/model.yaml
  routing/
    my-epp-profile.yaml

Catalog entries can be referenced by name, with the .yaml suffix optional:

manifesto render manifest model_provider/model --cluster my-context

If a file under clusters/ is named for the current kube context or kube cluster, render and deployment commands select it automatically, so --cluster can be omitted. Explicit paths and the existing MANIFESTO_CLUSTER and MANIFESTO_CLUSTER_MAP settings remain supported. Set MANIFESTO_CONFIG_HOME to override the entire user configuration path. User catalog entries take precedence over bundled entries with the same name.

Use the config commands to discover that effective catalog and validate a model/cluster pairing before rendering or deploying:

manifesto config home
manifesto config list models
manifesto config list clusters
manifesto config list routing
manifesto config resolve models deepseek-v4/1P-EP8-1D-EP8
manifesto config resolve routing wide-ep-lws-config
manifesto config edit models deepseek-v4/1P-EP8-1D-EP8
manifesto config export models deepseek-v4/1P-EP8-1D-EP8 -o model.yaml
manifesto config import models model.yaml --name experiments/model
manifesto config validate deepseek-v4/1P-EP8-1D-EP8 --cluster example-gb200

config list reports whether each winning entry is from the user or bundled catalog. Use --output name for shell scripts or --output json to also see when a user entry shadows a bundled entry. config validate loads both schemas, applies cluster defaults, and exercises the full renderer without contacting or changing the cluster.

config edit opens $EDITOR (or vi) on the user-catalog copy. Editing a bundled model first copies it and any relative extends parents into the user catalog, so the bundled source remains untouched. A new name creates a new user config. The edited file is schema-validated when the editor exits; an invalid file is retained so it can be fixed with the same command.

config export writes a self-contained YAML document with model inheritance flattened. config import validates and installs that portable form in the user catalog. Both refuse to overwrite files unless --force is passed.

Specs

Model specs live under models/.

Examples:

models/qwen/aggregated.yaml
models/deepseek-v4/1P-EP8-1D-EP8.yaml
models/deepseek-v4/3P-EP8-1D-EP16.yaml

A spec chooses the topology, roles, parallelism, vLLM args, and routing behavior:

release: wide-ep
topology: pd
accelerator: gb200

model:
  id: deepseek-ai/DeepSeek-V4-Pro
  image_ref: vllm.standard

vars:
  max_concurrency: 1024
  mtp_size: 1

roles:
  - name: decode
    workload_name: vllm-ep8-decode
    lws: {size: 4}
    parallelism: {tp: 1, pp: 1, dp: 16, ep: true}
    computed:
      env:
        MAX_TOKENS: max_concurrency
      vllm:
        max_num_batched_tokens: max_concurrency * mtp_size
        max_num_seqs: max_concurrency
        max_cudagraph_capture_size: max_concurrency

tp, pp, and dp are global tensor-, pipeline-, and data-parallel sizes. Each engine replica consumes tp × pp GPUs. Local model/DP groups, port fanout, and per-pod launch arguments are derived from the workload size and GPUs per pod. GPUs per pod is inferred from the parallel layout and the cluster profile; set parallelism.gpus to override it. Single-node roles render as Kubernetes Deployments; roles spanning multiple nodes render as LeaderWorkerSets. Set a role's workload to deployment or leaderworkerset to override that default. An explicit one-node LeaderWorkerSet can be useful when an admission controller integrates with LeaderWorkerSet rather than Deployment. Multi-node roles cannot select Deployment.

For example, this runs one TP2 × PP2 engine across four GPUs. Increase lws.size and divide those model-parallel GPUs evenly across pods to span nodes; Manifesto renders the native vLLM node-rank and headless-worker flags.

roles:
  - name: decode
    lws: {size: 1}
    parallelism: {tp: 2, pp: 2, dp: false}

Configure PP only through parallelism.pp. Manifesto rejects pipeline_parallel_size in vllm:, computed vLLM arguments, or vllm_raw_args so GPU allocation and the vLLM worker topology cannot disagree.

Unknown role keys are rejected at load time, so typos fail loudly instead of being silently ignored.

Keep EPP scheduling policy in the routing catalog, separate from model specs. A routing profile is an EndpointPickerConfig, for example routing/kv-aware.yaml:

apiVersion: llm-d.ai/v1alpha1
kind: EndpointPickerConfig
plugins:
  - {type: approx-prefix-cache-producer, name: gpu-prefix-cache-producer}
  - type: prefix-cache-scorer
    parameters: {prefixMatchInfoProducerName: gpu-prefix-cache-producer}
schedulingProfiles:
  - name: default
    plugins:
      - {pluginRef: prefix-cache-scorer, weight: 3}

Choose the profile when rendering or deploying; the model remains independent of the routing policy:

manifesto render manifest deepseek-v4/1P-EP8-1D-EP8 \
  --routing-profile wide-ep-lws-config
manifesto deploy deepseek-v4/1P-EP8-1D-EP8 \
  --routing-profile wide-ep-lws-config

Profiles resolve from the user catalog first and then bundled routing/ files. Manifesto embeds the resolved profile in the rendered EPP ConfigMap, so the Kubernetes artifact remains self-contained. Set MANIFESTO_ROUTING_PROFILE to choose a default without repeating the flag. Advanced specs can still use plugin_configs with plugins_config_file to bundle and select multiple files; existing routing.plugin_config specs continue to render as plugins.yaml.

Routed specs use a standalone Envoy sidecar in the EPP pod by default. The EPP Service exposes port 80, while the InferencePool remains the source of model endpoints and rank ports. Select the Gateway API frontend explicitly when shared Gateway infrastructure is required:

routing:
  kind: pd
  target_role: decode
  frontend: gateway  # standalone (default) or gateway

manifesto deploy removes resources left by the previous frontend after a successful apply. The low-level manifesto file apply command intentionally applies only the saved file and cannot prune resources omitted from it; use manifesto stop before that workflow when changing frontends.

workload_name is optional. When set, it controls the Deployment or LeaderWorkerSet name and therefore the Kubernetes pod name prefix, while full release-specific instance labels still scope routing and selectors.

Image versions are centralized in config/images.yaml. Model specs should use model.image_ref, with vllm.standard as the shared vLLM image reference, unless a spec needs an explicit one-off model.image.

Each cluster profile declares its available accelerator profiles and a default. Specs inherit the selected cluster's default unless they set accelerator. The selected entry controls accelerator allocation, accelerator-specific cache paths, and the development build architecture.

For model authors, accelerator allocation is cluster-owned: model specs keep the same role GPU counts whether the cluster uses extended resources or Dynamic Resource Allocation (DRA). Only cluster operators select or change the backend.

An extended-resource profile declares that backend explicitly:

accelerators:
  default: b200
  profiles:
    b200:
      allocation:
        extended_resource:
          resource_name: nvidia.com/gpu
      presence_label: nvidia.com/gpu.present
      gpu_arch: b200
      torch_cuda_arch_list: "10.0+PTX"

To use DRA instead, the cluster operator changes only allocation:

    b200:
      allocation:
        dra:
          device_class_name: gpu.nvidia.com
      presence_label: nvidia.com/gpu.present
      gpu_arch: b200
      torch_cuda_arch_list: "10.0+PTX"

allocation must contain exactly one of extended_resource or dra.

DRA operator requirements

A DRA profile renders one resource.k8s.io/v1 ResourceClaimTemplate for each GPU-bearing role. Each template requests the role's GPU count with ExactCount, and only the model container receives the resulting claim. CPU, memory, ephemeral storage, and RDMA remain ordinary container resources. Template names include a digest of the complete immutable claim specification, so a release can safely reuse a stable workload name when its claim metadata changes.

The cluster must serve resource.k8s.io/v1 and provide the configured DeviceClass. When Kueue is enabled, configure its DRA integration and a deviceClassMappings entry for that class, then give the mapped logical resource ClusterQueue quota. Manifesto verifies the DRA API and DeviceClass before deployment; Kueue remains authoritative for mapping and quota accounting.

Manifesto applies shared vLLM arguments before role-specific vllm: and computed arguments. By default, Uvicorn access logs omit the high-frequency /health, /v1/models, and /metrics endpoints while preserving inference request logs. A role can replace the excluded endpoint list with vllm.disable_access_log_for_endpoints, or set it to null to omit the Manifesto default entirely.

Feature contracts

Role settings activate typed serving features before rendering. Data, pipeline, and expert parallelism, P/D, and llm-d are features. Connector and all-to-all implementations are reported separately as backends; workload kind and platform resources are derived consequences. For example:

  • routing.kind: disabled selects direct vLLM, while load_aware or pd selects llm-d.
  • data parallelism uses external DP when llm-d is selected and internal DP for direct vLLM.
  • P/D implies llm-d and a routing proxy for now.
  • a NIXL connector backend supplies VLLM_NIXL_SIDE_CHANNEL_HOST from the Pod IP unless the role provides an explicit value; other connector backends are preserved without being treated as NIXL.
  • a parallel shape spanning multiple nodes selects a LeaderWorkerSet; single-node shapes select a Deployment.
  • DeepEP is reported as the selected all-to-all backend for EP, and an IMEX claim is derived from the cluster platform capability when configured.

Dependencies are resolved transitively and the registry rejects implication cycles. They do not add framework annotations or dependency metadata to the rendered objects. Inspect the internal resolution separately:

manifesto explain models/deepseek-v4/1P-EP8-1D-EP8.yaml \
  --cluster clusters/example-gb200.yaml \
  --user "$USER"

The explanation includes each role's enabled features, selected backends, workload and fabric profile, environment provenance, and resource claims.

Lower-level integrations can append exact command-line fragments with vllm_raw_args. Unlike the vllm: mapping, these strings are not renamed, quoted, or otherwise interpreted, and they are emitted after structured and computed arguments. Each entry is trusted shell syntax, so quote spaces and shell metacharacters within the entry when needed:

roles:
  - name: decode
    vllm_raw_args:
      - --trust-remote-code
      - --attention-config.backend=FLASH_ATTN

Specs can also extend a base YAML file and override only the fields that differ:

extends: wide-ep-base.yaml
release: wide-ep-2p-ep8-1d-ep8

roles:
  prefill:
    lws: {replicas: 2}

Overrides deep-merge mappings. roles may be written as a map keyed by role name in override files; each role override is merged into the matching base role before normal schema validation.

Every deployment includes an idle-shutdown controller by default. Once all expected vLLM API servers are ready, it watches request counters and running or queued requests across all roles. After 45 idle minutes it scales the model workloads and endpoint picker to zero, deletes the instance Gateway when the Gateway frontend is selected so its platform-managed proxy replicas are released, and scales itself to zero. Applying or deploying the spec again restores all of those resources. Change the timeout or opt out in the runtime configuration:

runtime:
  idle_shutdown:
    timeout_minutes: 90
    # enabled: false

Metric or Kubernetes API failures reset the idle timer, so the controller does not shut down an instance when it cannot establish that the instance is idle. For one-off operations, the CLI can override either setting without editing the model spec:

manifesto deploy models/qwen/qwen3-0.6b.yaml \
  --context example-context \
  --idle-timeout 15m

manifesto render manifest models/qwen/qwen3-0.6b.yaml \
  --no-idle-shutdown

DeepSeek V4 wide-EP specs use filenames that encode only the parallel layout: <prefill-replicas>P-EP<width>-<decode-replicas>D-EP<width>.yaml. For example, 3P-EP8-1D-EP16.yaml means three prefill LWS replicas at EP8 and one decode LWS replica at EP16. Backend choices stay inside the YAML because optimal backends can change independently of the parallel layout.

Cluster Profiles

Cluster profiles live under clusters/.

Three synthetic profiles are included for documentation and tests:

clusters/example-gb200.yaml
clusters/example-h200.yaml
clusters/example-stateless-b200.yaml

They deliberately omit real network, storage, scheduling, and provider details. Put usable profiles in the private user catalog described above; clusters/*.yaml is ignored by git except for files named example-*.yaml.

The stateless example intentionally declares no cache or logging filesystem. Its model pods omit persistent cache env, storage mounts, and the log tee wrapper; libraries use container-local defaults and logs stream to stdout. An external vllm-envs worktree cannot be selected unless its absolute path is covered by a model-pod volume mount.

Manifesto also omits Kubernetes fields whose defaults are sufficient. Set pod_defaults.image_pull_secrets, image_pull_policy, termination_grace_period_seconds, working_dir, or container_security_context only when cluster or image policy requires an explicit value. Pull secrets are existing Secret names in the workload namespace; Manifesto references them but does not create credential data. They are attached to every generated PodSpec, including routing and idle-shutdown pods. For example:

pod_defaults:
  image_pull_secrets:
    - example-registry-credentials

A dedicated model-server ServiceAccount is generated only when an OpenShift SCC must be bound to it.

Cluster profiles own environment that should not be repeated in every model spec:

  • available accelerators and the cluster default
  • GPUs per node
  • shared and local volume mounts
  • optional namespace bootstrap configuration for shared PVCs
  • pod annotations, scheduling constraints, extra devices, and security context
  • RDMA extended-resource requests and OpenShift SCC authorization
  • user, log, and cache path templates
  • llm-d release
  • UCX/NCCL/NVSHMEM/IMEX fabric env profiles
  • optional Kueue LocalQueue selection for GPU LeaderWorkerSets

When a role omits resources.cpu or resources.memory, Manifesto derives the request from the role's inferred GPUs per pod (N) using built-in defaults:

  • CPU: 6 + 2N
  • memory: max(64Gi * N, 128Gi)

For example:

GPUs per pod CPU Memory
1 8 128Gi
2 10 128Gi
4 14 256Gi
8 22 512Gi

Set either value directly on a role when a model needs a different request:

roles:
  - name: decode
    parallelism: {tp: 4}
    resources: {memory: 224Gi}

Explicit role resource values always win independently, so the example keeps the built-in 14-CPU request while using exactly 224Gi of memory.

Reusing cluster workload settings

Controllers that submit other GPU workloads can reuse the cluster-owned part of a Manifesto profile without depending on model topology or routing:

from manifesto.cluster import load_cluster
from manifesto.workload import workload_settings

settings = workload_settings(load_cluster("clusters/my-cluster.yaml"))
accelerator = settings.accelerator("gb200")

The portable projection contains accelerator resource or DeviceClass names and node selectors, the default Kueue LocalQueue, accelerator-attached platform claims such as an NVIDIA compute-domain channel, and pod placement defaults such as affinity, tolerations, DNS, annotations, and image pull policy. The projection carries only each claim's portable name and template reference; storage, the rest of the fabric configuration, and launch settings remain part of Manifesto's serving-specific cluster model.

Manifesto also owns a controller-neutral workload IR and its Kubernetes object lowering. This lets tools describe a pod template and lifecycle policy while Manifesto consistently applies cluster placement and Kueue metadata:

manifesto render workload examples/non-indexed-job.yaml \
  --cluster clusters/my-cluster.yaml \
  --accelerator gb200

The job backend emits an ordinary, non-indexed batch/v1 Job by default and can explicitly describe Indexed Job completion policy for distributed work. A workload may optionally declare a normal or headless Service. The same IR lowers Deployment and LeaderWorkerSet backends, including explicit LWS leader templates; Grove is reserved as an extension point until its rendering contract is implemented.

An explicit LWS leader template is intentionally same-shaped: Manifesto applies the workload's accelerator count, accelerator container, and cluster-owned claims to both leader and worker templates. It also removes Kueue queue labels from both Pod templates and keeps queue selection on the LeaderWorkerSet, as required by Kueue's LWS integration.

Kueue admission

GPU roles can be admitted through a Kueue LocalQueue selected in the cluster profile:

kueue:
  local_queue: example-gpu-queue

When configured, Manifesto preserves each role's native workload kind. A single-node role remains a Deployment and receives kueue.x-k8s.io/queue-name on both the Deployment and its pod template. Each Deployment pod is therefore admitted independently. A role whose topology requires a LeaderWorkerSet receives the queue label on the LeaderWorkerSet, whose complete leader/worker group is admitted atomically. The queue is cluster- and namespace-specific operational configuration, so it belongs in the private cluster profile rather than a shareable model spec. Omitting kueue preserves the same workload kinds without admission metadata. CPU-only supporting resources, including the in-cluster end-to-end probe Job, do not receive Kueue metadata or suspension.

Before applying a queued workload, manifesto deploy verifies that the Kueue APIs are served, the selected LocalQueue is Active=True, and its referenced ClusterQueue is Active=True. It also prints every init-container and container request in each rendered workload pod template so quota requirements are visible before mutation, and rejects request resource names that the ClusterQueue does not cover. Deployments do not require the LeaderWorkerSet API; native LeaderWorkerSets are checked for it.

Changing queue admission on an existing LeaderWorkerSet is intentionally a recreate operation because its admitted group is atomic. Enabling, changing, or disabling its queue deletes it before applying the replacement. Deployment queue changes use the Deployment rollout and replace independently admitted pods without changing workload kind. Moving a same-named role between native Deployment and LeaderWorkerSet shapes deletes the obsolete controller first. LWS recreation and kind transitions interrupt serving; use a distinct release name for a parallel, zero-downtime rollout.

The workflow CLI requires a cluster profile for commands that render a spec. Set MANIFESTO_CLUSTER directly, pass --cluster, or set MANIFESTO_CLUSTER_MAP in .env as a local lookup table keyed by kube context or kube cluster name:

MANIFESTO_CLUSTER=clusters/example-gb200.yaml manifesto deploy models/qwen/aggregated.yaml
manifesto deploy models/qwen/h200-aggregated.yaml

The namespace defaults to the selected kube context namespace, falling back to default when the context has no namespace. Use --context to select a context without changing kubectl's current context, or set MANIFESTO_NAMESPACE in .env to override the namespace.

Shared storage is configured without assuming a filesystem implementation:

storage:
  shared_volume:
    persistentVolumeClaim:
      claimName: model-cache
  shared_mount_path: /mnt/shared
  local_nvme_path: /mnt/local

shared_volume accepts any Kubernetes volume source, such as persistentVolumeClaim, hostPath, csi, nfs, or emptyDir. Profiles backed by separate host-local cache volumes can omit it entirely.

When Manifesto should provision a referenced shared PVC in each namespace, declare storage.shared_claim with its storage class, access modes, and size, inspect it with manifesto render bootstrap, then run manifesto deploy bootstrap. The apply command uses kubectl apply, so it is safe to repeat. PVCs managed outside Manifesto should omit shared_claim.

Set platform: openshift for OpenShift clusters. This adds a stable USER fallback for Python libraries when containers run under an arbitrary UID, without affecting standard Kubernetes profiles. It also omits the node-exporter sidecar because its /sys and /proc hostPath volumes are incompatible with OpenShift's restricted SCC; dcgm-exporter remains enabled when requested. An SCC can be granted to each release-specific model service account:

openshift:
  scc: custom-scc

A model role can override automatic fabric selection when its backend requires one explicitly:

roles:
  - name: decode
    fabric_profile: custom_ep

Fabric profiles provide backend-specific environment settings:

fabric:
  profiles:
    deepep_decode:
      env:
        VLLM_USE_NCCL_SYMM_MEM: "1"

Select a non-default Gateway API implementation in the cluster profile:

gateway:
  class_name: data-science-gateway-class
  service_type: ClusterIP

Compiled-cache paths derive their cache key from the resolved model image tag or digest. Custom and dev builds can force a fresh namespace explicitly:

cache:
  key: dev-build-42

When persistent cache storage is configured, model pods clear their JIT and compilation caches after a failed process exit, or when the same container restarts after terminating without running its exit handler. Stateless pods omit this machinery. FlashInfer autotuning results under $VLLM_CACHE_ROOT/flashinfer_autotune_cache are preserved. Disable this default for a deployment only when retaining possibly corrupt compiled artifacts is intentional:

cache:
  cleanup_on_crash: false

Model-server stdout/stderr is persisted by the generated launch script. Configure the backing PVC and root path in the cluster profile:

logging:
  pvc: example-shared-cache
  mount_path: /mnt/shared
  root: /mnt/shared/{user}/logs

The rendered pods tee logs to {root}/{role}, for example /mnt/shared/<user>/logs/decode and /mnt/shared/<user>/logs/prefill.

Manual Manifest Workflow

The file workflow is the preferred path when you want to share or tweak exactly what will be deployed:

manifesto render file models/deepseek-v4/1P-EP8-1D-EP8.yaml
$EDITOR /tmp/manifesto.yaml
manifesto file diff
manifesto file apply

manifesto file diff compares the rendered file against the live cluster with kubectl diff. manifesto file apply applies that file as-is. Use -o/--output with render file, or set MANIFESTO_RENDER_OUT, to choose a different file.

The low-level manifesto render manifest command performs no cluster I/O when passed an explicit --cluster and --namespace (or with MANIFESTO_NAMESPACE set). Workflow commands such as deploy, ready, file diff, and file apply are the cluster-touching commands.

Lifecycle Commands

manifesto deploy SPEC *ARGS          # render and apply a full stack
manifesto servers                    # list live servers in the namespace
manifesto stop [SPEC] [--now]        # discover and delete live objects
manifesto stop --instance ID         # stop by live instance identity
manifesto ready SPEC                 # wait for pods and routing frontend
manifesto test e2e SPEC              # fresh-namespace dev + inference integration test
manifesto deploy routing SPEC *ARGS  # update routing only

manifesto stop is stateless: it finds live objects through their Manifesto instance labels and does not depend on a stored manifest or deployment inventory. With no target it opens an fzf picker, or a numbered picker when fzf is unavailable. Selecting an entry starts teardown immediately. Explicit spec and --instance targets remain noninteractive and are safe for scripts.

Teardown removes routing, model workloads, supporting objects, and remaining instance pods. It does not drain active requests or delete shared logs, caches, or model storage. --now retains the force-delete behavior.

Enable parser-driven completion for commands, flags, model configs, cluster and routing profiles, accelerator names, live deployment instance IDs, and local paths:

source <(manifesto completion zsh)   # zsh
source <(manifesto completion bash)  # bash
manifesto completion fish | source  # fish

For scripts, manifesto servers --output name prints one instance ID per line, and --output json includes the exact resources associated with each instance.

Slow clusters

manifesto servers and manifesto stop find live objects by listing every Manifesto-managed resource type. Those lists run concurrently, one request per type, so discovery costs roughly one API round trip of wall-clock time instead of one per type. There is no kubectl api-resources preflight: a type the cluster does not serve simply lists as zero objects, and pruning the set in advance would not save any wall clock once the lists run in parallel.

Discovery reads are bounded and retried on transient faults. Tune them when an API server is unusually slow or flaky:

MANIFESTO_KUBECTL_TIMEOUT=300  # seconds per read attempt; 0 disables the bound
MANIFESTO_KUBECTL_RETRIES=4    # extra attempts after a transient failure
MANIFESTO_TRACE=1              # log every kubectl invocation and its duration

MANIFESTO_TRACE=1 is the fastest way to find which call is slow:

MANIFESTO_TRACE=1 manifesto stop --instance ID

Shell completion of live instance IDs uses a short, non-retrying timeout, so a slow cluster yields no candidates instead of stalling the prompt.

Examples:

manifesto deploy models/qwen/aggregated.yaml
manifesto ready models/qwen/aggregated.yaml
manifesto test e2e models/qwen/qwen3-0.6b.yaml --cluster my-cluster
manifesto deploy models/deepseek-v4/1P-EP8-1D-EP8.yaml \
  --vllm-env /mnt/shared/$USER/vllm-envs/feature
manifesto deploy routing models/deepseek-v4/1P-EP8-1D-EP8.yaml
manifesto stop

manifesto test e2e owns the complete test lifecycle. By default it creates a fresh namespace, replaces profile storage with emptyDir volumes, deploys the vLLM bundled in the model image, waits for it to become ready, and creates an unprivileged Job that calls /v1/models followed by a real /v1/completions request. The Job targets the standalone router Service or generated Gateway when routing is enabled, and the model Service otherwise.

Pass --vllm-env /absolute/worktree/path to test an existing vllm-envs worktree instead. That mode preserves the cluster profile's normal storage so the worktree remains visible; its volume references must be usable in the fresh test namespace. Manifesto never creates, synchronizes, or modifies the worktree.

The namespace is deleted at the end, including on failure. Pass --namespace to choose its name (the command refuses an existing namespace), --keep-namespace to retain all test resources for inspection, --timeout to change the inference deadline, or --image to use a mirrored Python image. HF_TOKEN and a cluster profile are required, as with normal deployments.

Every rendered object is scoped by the release-based instance identity:

{release}

Names, labels, and selectors derive from that identity. User-scoped storage paths and the llm-d.ai/owner label still retain the user. Deployments must have unique release names within a namespace. Clusters whose users share release names can enable user-prefixed identities in their profile:

naming:
  user_prefix: true

External vLLM Environments

Use vllm-envs to create, synchronize, build, and remove development environments. Manifesto only points model pods at an existing vLLM worktree whose .venv has already been initialized:

manifesto deploy models/deepseek-v4/1P-EP8-1D-EP8.yaml \
  --vllm-env /mnt/shared/$USER/vllm-envs/feature

The path must be absolute and covered by a volume mount in every model pod. Manifesto fails before launching vLLM if the worktree or .venv/bin/activate is missing. Set the same pointer declaratively with runtime.vllm_env in a model spec. Without either setting, the deployment uses vLLM from the model image.

Monitoring

The monitoring/ directory contains namespace-scoped Prometheus and Grafana configuration. Per-pod exporters such as DCGM exporter are part of rendered model manifests and can be toggled from the model spec.

Requirements

Local tools:

  • uv
  • kubectl

Expected .env values:

HF_TOKEN=replace-me
KUBECONFIG=/path/to/kubeconfig
MANIFESTO_CLUSTER_MAP=my-context=clusters/example-h200.yaml
MANIFESTO_NAMESPACE=workload-ns  # optional, defaults to current kube namespace

manifesto deploy and manifesto file apply create or update the namespace's hf-secret from HF_TOKEN before launching pods. The token is sent to kubectl apply over stdin and is not included in rendered workload files. Pure manifesto render commands remain side-effect free; manage the secret explicitly when applying rendered stdout with kubectl.

Test

uv run --extra test pytest
manifesto --help

Design Bias

Manifesto is intentionally not a controller, operator, CRD, or matching engine.

It is a client-side renderer:

model spec + cluster profile + user -> raw Kubernetes YAML

The YAML is the artifact. Inspect it, edit it, share it, apply it.

About

Declarative Kubernetes renderer for llm-d and vLLM deployments

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages