You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Every environment variable read by the coordinator, the provider CLI
(darkbloom), console-ui and admin-ui: accepted values, the compiled default,
the code that reads it, and its effect. Defaults are the fallbacks at the cited
symbol; a production or dev host may pin a different value in its environment
file. Secrets are named, never valued. Unless a row says live, the variable is
read once at process start and a restart applies a change.
App Attest shadow configuration defines evidence collection and receipt renewal. Provider authorization defines the separate serving and MDM-removal opt-ins, both disabled by default. The account cohort, safe-version floor and qualified build/code hashes remain required. Durable build approvals replace per-release env edits; existing env pairs are a bootstrap fallback that cannot override a durable revocation. Shadow alone grants no trust; an explicitly enabled qualified App Attest path can replace legacy serving verification.
Signal/AppKit and planned metadata-reconnect drain in provider-swift/Sources/ProviderCore/Service/ProviderTermination.swift (timeoutSeconds); launchd environment allowlist preserves it
A CLI deadline expiry leaves a running, non-admitting process and disables
watchdog/login restart. Signal-only shutdown preserves configured login startup. See lifecycle commands
for recovery and explicit force semantics. Existing loaded launchd jobs must be
restarted to adopt the new allowance. Local mailbox files are owner-only under
lifecycle/ beside the daemon state file and bind PID plus kernel process-start
time; they are not network control endpoints or serving credentials.
Where values are set
Component
Where the process gets its environment
Coordinator, production
/etc/d-inference/env (root-only, boot disk) on the Confidential VM. Secrets are placed by hand; deploy/gcp/prod/refresh-env.sh runs before Docker at every boot, adds any key from deploy/gcp/prod/release-env-defaults that is absent, migrates a few exact historical values, never overwrites an operator-set value, and refuses to run when a required key is missing or empty. The list of keys production must have is maintained once in ../operations/coordinator-deploy.md#environment-file. The container entrypoint coordinator/deploy/start.sh reads USER_PERSISTENT_DATA_PATH, MICROMDM_API_KEY, MDM_PUSH_P12_B64, DOMAIN and EIGENINFERENCE_MDM_WEBHOOK_SECRET itself before it execs the coordinator binary; everything else is read by Go code.
Whatever shell exports go run ./coordinator/cmd/coordinator inherits. EIGENINFERENCE_ALLOW_MEMORY_STORE=true is the only way to start without a database.
Provider CLI, darkbloom start --foreground
The invoking shell's environment, minus the 13 variables scrubbed by provider-swift/Sources/ProviderCore/Security/EnvironmentScrubber.swift. Every DARKBLOOM_* row below applies.
Provider CLI, installed LaunchAgent
darkbloom start (daemon mode) writes a launchd plist whose EnvironmentVariables are built by passthroughEnvironment in provider-swift/Sources/ProviderCore/Service/LaunchAgent.swift: only the allow-list passthroughEnvKeys + inferencePassthroughEnvKeys is copied from the operator's shell (DARKBLOOM_DRAIN_TIMEOUT_SECONDS, DARKBLOOM_PREFIX_CACHE, DARKBLOOM_PREFIX_CACHE_MEMORY, DARKBLOOM_MLX_RESOURCE_DEBUG, DARKBLOOM_CBV2_PAGED_KV, DARKBLOOM_CBV2_MTP, DARKBLOOM_MTP_MAX_RECTANGULAR_TOKENS, DARKBLOOM_KV_BACKEND_GUARD, DARKBLOOM_MLX_CACHE_LIMIT_GB, DARKBLOOM_MLX_MEMORY_RESERVE_GB, DARKBLOOM_CBV2_MAX_PARTIAL_PREFILLS, DARKBLOOM_PREFILL_DEADLINE_MODE), plus MLX_GATHER_QMM_EXPERT_SLICES only when it is exactly 1. PATH is deliberately dropped. The watchdog plist (provider-swift/Sources/ProviderCore/Service/WatchdogAgent.swift) additionally forwards DARKBLOOM_NO_UPDATE_CHECK. Every other provider variable is inert under launchd.
Provider CLI, provider.toml
~/.config/darkbloom/provider.toml (ConfigManager in provider-swift/Sources/ProviderCore/Config/ProviderConfig.swift) is the durable configuration; a variable that overrides a config key says so in its Effect cell (DARKBLOOM_CBV2_PAGED_KV, DARKBLOOM_CBV2_MTP, DARKBLOOM_MLX_MEMORY_RESERVE_GB, DARKBLOOM_GEMMA4_PREFILL_CHUNK_EVAL).
console-ui
Next.js .env* files or the hosting build environment (Vercel-style). Every console-ui variable is NEXT_PUBLIC_* or build-tooling: inlined at build time, so changing one requires a rebuild. There is no server-only secret; a gitignored .env.local in console-ui/ is the only local file and no .env.example exists.
admin-ui
Server-only runtime variables read by React Server Components on each request; set them in .env* or the host environment. NODE_ENV is set by Next.
Serves net/http/pprof on a separate listener; bind loopback or firewall it. A successful listener enables mutex sampling at fraction 100 and block sampling at rate 1_000_000 ns (enableContentionProfiling).
Replaces the store-built runtime manifest at boot; discarded by the next release registration or deactivation, which rebuilds the union from active releases.
Routing, admission and TTFT
Trust floor, model routing and per-request quality:
Enables the request-absolute SLA only for selected authenticated accounts. Email matching is case-insensitive and uses the stored user, never headers or the service role. Verify the production selector before rollout; a verified account ID avoids dependence on email changes.
For selected accounts, overrides both SLA terms for exact model IDs; an explicit public alias policy takes precedence over the resolved build. Retains one second of coordinator headroom. Base 1001–600000 ms, slope 0–100 ms/token; invalid/duplicate entries fail startup atomically. Applied after the legacy base table.
coordinator/registry/health_ejection_switch.go (healthEjectionSwitch, parsed once at package init); coordinator/registry/health_ejection.go (healthEjectionEnabled)
Live first-content deadline base for selected accounts (FirstContentDeadlineBase, plus 1 ms per prompt token); legacy base-only policy may tighten it; an explicit model SLA overrides both terms.
Spill no_provider requests into the queue when an idle on-disk provider can be warmed, and kick the load.
Capacity breakers:
Variable
Values / type
Default
Read in
Effect
EIGENINFERENCE_BUDGET_CLAMP
bool
true
coordinator/registry/budget_clamp.go
Clamp admission to a provider whose reported token budget is stale after a capacity 503.
EIGENINFERENCE_BUDGET_CLAMP_TTL_SECONDS
seconds
300
coordinator/registry/budget_clamp.go
Fail-open bound on how long a clamp can hold.
EIGENINFERENCE_CAPACITY_COOLDOWN_THRESHOLD
integer (0 disables)
5
coordinator/registry/capacity_cooldown.go
Consecutive capacity rejects before a (provider, model) pair is cooled down.
EIGENINFERENCE_CAPACITY_COOLDOWN_WINDOW_SECONDS
seconds
60
coordinator/registry/capacity_cooldown.go
Window in which rejects count toward the threshold.
EIGENINFERENCE_CAPACITY_COOLDOWN_TTL_SECONDS
seconds
120
coordinator/registry/capacity_cooldown.go
Initial cooldown; doubles on each failed probe.
EIGENINFERENCE_CAPACITY_COOLDOWN_MAX_TTL_SECONDS
seconds
600
coordinator/registry/capacity_cooldown.go
Ceiling of the exponential cooldown.
EIGENINFERENCE_CAPACITY_RATE_PENALTY_MS
milliseconds (≤ 0 disables)
15000
coordinator/registry/capacity_rate.go
Scores a penalty proportional to a provider's recent capacity-reject rate.
Reservation commit lock:
Variable
Values / type
Default
Read in
Effect
EIGENINFERENCE_RESERVE_COMMIT_MODE
shared, global (trimmed, case-insensitive; any other value is treated as shared and logged as unknown)
shared
coordinator/registry/gate_commit_mode.go (loadReserveCommitMode, parseReserveCommitMode); read once at registry.New
How the reservation commit (commitProviderReservation, ReserveNextFromPlan) holds the registry lock. shared commits under r.mu.RLock plus the winner's p.mu; global is the kill switch that restores the fleet-wide r.mu.Lock() commit serialization. The fault-tracker recorders stay on their per-identity gates in both modes; see ../architecture/routing.md.
Quality concurrency cap:
Variable
Values / type
Default
Read in
Effect
EIGENINFERENCE_QUALITY_CONCURRENCY_CAP
bool
true
coordinator/registry/config.go (ReadConfig)
Per-provider admission cap derived from each model's quality concurrency instead of the flat cap.
EIGENINFERENCE_QUALITY_CONCURRENCY_OVERCOMMIT
float ≥ 0
1.2 (defaultQualityCapOvercommit; the 2.0 fallback in ReadConfig is replaced when the variable is unset)
Cache-aware routing (semantics in ../architecture/cache-aware-routing.md). refresh-env.sh seeds absent keys from deploy/gcp/prod/release-env-defaults — production ships MODE=off, PERCENT=1, MAX_PLAN_QPS=1 — and never overwrites a value an operator has set:
Restricts network cache participation before cohort/QPS/sidecar work; [] denies all. Invalid configuration refuses startup, including while mode is off.
EIGENINFERENCE_CACHE_ROUTING_PERCENT
float (0, 100]
100
coordinator/registry/config.go (envStrictFloat)
Share of eligible requests that use cache routing; malformed values refuse startup.
EIGENINFERENCE_CACHE_ROUTING_MAX_PLAN_QPS
float 0–1,000,000
0 (unlimited)
coordinator/registry/config.go (envStrictFloat)
Rate limit on cache-plan computation.
EIGENINFERENCE_CACHE_ROUTING_TTL
Go duration ≥ 0
10m
coordinator/registry/config.go
SSD holder lifetime; resident holders use the smaller of this value and cacheRoutingMemoryTTL = 30 * time.Second (coordinator/registry/cache_tiers.go, receiptTTL).
EIGENINFERENCE_CACHE_ROUTING_MAX_HOLDERS
integer 1–32
4
coordinator/registry/config.go
Maximum machines per exact content prefix and tier, across provider epochs.
Keys the affinity digests so raw identity and prefix bytes are never stored.
The optional artifact array uses exactly model_id, model_aggregate_sha256
and prompt_contract_id string fields. Model IDs match resolved catalog IDs
exactly (at most 512 bytes, no wildcard, surrounding whitespace or control
separators); both hashes must be 64 lowercase hexadecimal characters. Missing,
unknown or duplicate fields, duplicate tuples, null, a blank setting, trailing
JSON and oversized input are configuration errors. Multiple exact revisions of
one model may be listed as distinct tuples. An absent variable preserves the
existing unrestricted eligibility; [] is an explicitly configured empty list.
MODE=off still disables participation with any list.
The list is parsed once at startup and compiled into immutable membership;
ConfigureCacheRouting replaces it with the holder/attempt tracker. Exclusion
returns ordinary ineligible without a reusable remote scope or inference
error. It does not alter provider capabilities or local HTTP cache policy
(coordinator/registry/cache_route_keys.go, PlanCacheRouteWithResult). No
artifact entries are populated by the shipped defaults; follow the
rollout runbook to configure a measured
artifact subset before activating network routing.
Rate limits and service-account admission (coordinator/ratelimit/config.go, ReadConfig):
Enables new international onboarding, quotes and withdrawals. Production refresh preserves an explicit false and requires the funding account and webhook secret before activation; runtime validation also requires the base EIGENINFERENCE_STRIPE_SECRET_KEY used for Connect. Reconciliation continues with configured credentials even when disabled.
Public R2 bucket URL release binaries are pulled from; release registration is refused (503) until it is set, and every registered artifact URL must live under it. See ../operations/release-policy-rollout.md.
Prompt sidecar and media fetch
Prompt sidecar (coordinator/promptcontract/config.go, ReadSupervisorConfig; every duration is an integer count of milliseconds; Check refuses startup on an out-of-range value when the sidecar is enabled). Semantics: ../architecture/prompt-contract-sidecar.md.
Variable
Values / type
Default
Read in
Effect
EIGENINFERENCE_PROMPT_SIDECAR_ENABLED
bool
false
coordinator/promptcontract/config.go
Starts the supervised prompt-contract sidecar and its artifact provisioner.
Sidecar-side concurrency, connection and loaded-contract limits.
EIGENINFERENCE_PROMPT_SIDECAR_MEMORY_LIMIT_MIB
MiB ≥ 256
1024
coordinator/promptcontract/config.go
Memory ceiling applied to the sidecar process.
Media fetch (coordinator/mediafetch/config.go, ConfigFromEnv; a set-but-unparseable value is recorded and fails startup in Check, so a mistyped kill switch cannot silently keep fetching):
Variable
Values / type
Default
Read in
Effect
EIGENINFERENCE_MEDIA_FETCH_ENABLED
bool
true
coordinator/mediafetch/config.go
Lets the coordinator download image_url/video_url inputs and inline them as data: URIs before relaying; when off, requests carrying remote media are rejected. See ../architecture/data-flow.md.
Fraction of successful requests the profiler samples; slow, failed and retried requests are always recorded.
Provider CLI (darkbloom)
Parsing convention: affirmative values are 1/true/yes/on, negative values 0/false/no/off, case-insensitive. Only the variables named in the LaunchAgent allow-list above reach an installed daemon; everything else applies to darkbloom start --foreground and to the benchmark and test binaries.
Operator-facing: daemon, paths, updates
Variable
Values / type
Default
Read in
Effect
DARKBLOOM_NO_UPDATE_CHECK
any value
unset
provider-swift/Sources/darkbloom/Darkbloom.swift; provider-swift/Sources/darkbloom/StartCommand+Modes.swift; provider-swift/Sources/darkbloom/WatchdogCommand.swift; provider-swift/Sources/ProviderCore/ProviderLoop+AutoUpdate.swift; forwarded by provider-swift/Sources/ProviderCore/Service/WatchdogAgent.swift
Skips the startup version banner, the in-daemon auto-update loop, the start-mode check and the watchdog's update check; scripts/install.sh sets it for the runtime smoke test.
Kill switch for paged KV; beats the provider.toml setting. The owned Flash-Next candidate joins the exact automatic policy; a default is not runtime qualification.
Optional assertion for resolved paged storage; a nonempty value must match every measured native layer. Unsupported values or mismatches refuse explicit paged construction.
Trusted-history replay may compute only the embedded assistant's K/V. Prefix save/restore uses the separate typed history codec. Not forwarded to LaunchAgents.
Upper proposal limit for adaptive depth; invalid selected limits fall back to seven. This is not a fixed proposal count. Not forwarded to LaunchAgents.
These controls belong to the private candidate contract,
not a public catalog activation. They are read in foreground/local processes;
these model-specific controls are not forwarded by LaunchAgent.inferencePassthroughEnvKeys.
Installed processes still use the source defaults.
Variable
Values / type
Default
Read in
Effect
DARKBLOOM_QWEN4_MODEL_PATH
absolute existing native Qwen4 snapshot directory with config and weight index
Selects an isolated directory only for exact DarkBloom/Qwen3.8-Flash-Next-Q4-mtp. Scanner and load resolution agree. An invalid explicit override hides/refuses this model instead of using its old cached snapshot; other IDs are unchanged. Normal artifact hashing, admission and runtime checks remain active. No HOME or cache mutation.
DARKBLOOM_QWEN4_LISTING_CONTEXT
positive integer, lower-only
positive native context; 262_144 fallback for the qualified IDs without metadata
Bounds prompt plus resolved output reservation. Unset, empty, invalid, zero and negative values retain native capacity; a positive override can only lower it. Unknown native artifacts without metadata have no invented fallback. Coordinator SLA policy and physical-memory admission remain independent.
Immutable learned PLE weights remain SSD-backed independently of request prefix caching. Keep this enabled for candidate qualification; disabling request caching does not disable PLE.
A valid explicit caller argument wins over the environment. Invalid explicit values retain the caller fallback. This changes scheduling, not arithmetic order, precision or MTP depth.
Early singleton text layer submission on valid native paged caches at widths 1–6. Media/explicit positions, wider/batched shapes and faulted or unknown caches retain the existing scheduling. Other explicit spellings stay disabled.
Native DiffusionGemma expert reduction
These controls affect native DiffusionGemma inference, not its weights,
denoising recipe or autoregressive MTP capability. Set overrides before starting
a foreground provider or benchmark. These variables are not forwarded by
LaunchAgent.inferencePassthroughEnvKeys; installed processes retain the source
default unless their own environment supplies an override.
Variable
Values / type
Default
Read in
Effect
DARKBLOOM_DIFFUSION_EXPERT_UNSORT
unset enables; case-insensitive 1, true, yes, on enable; any other explicit value disables
Reuses the ordered weighted-unsort kernel for sorted BF16 outputs/weights with hidden size 2816, eight selected experts, at least 64 assignments and a matching uint32 inverse permutation on the ordinary GPU device/stream. Flattened output addresses must fit the shader's uint32 range, with checked host multiplication. Avoids materializing the restored expert-output intermediate. Training, CPU/custom streams, oversized shader indices, other shapes/dtypes and explicit rollback retain the original scatter/multiply/reduce graph. No context-capacity, sampler, attention or precision change.
DARKBLOOM_DIFFUSION_SOFT_EMBEDDING
case-insensitive 1, true, yes, on enable; unset or any other value disables
Uses the existing non-transposed affine matrix math with a 64-row GPU tile for the 256-by-262144 soft-conditioning input and 2816-wide Q8/group64 embedding. Native BF16 inference on the ordinary GPU stream only; training, traced/retained graphs, other geometry/quantization and missing resources retain original quantizedMM. Input-view preparation, weights and native sampler are unchanged.
DARKBLOOM_DIFFUSION_COMPILED_SAMPLER
exact 1 enables; unset or any other value disables
Compiles native entropy/acceptance and sampling-state operations for the eligible single-row, 256-position, 262144-vocabulary FP32 path. Preserves key ordering, integer draws, exact RNG/clamp constants and validation before state mutation; arbitrary callbacks, CPU/custom streams, other geometry/storage and nondefault stability/entropy/confidence policies retain the original sampler. Request arrays are explicit inputs to bounded compiled variants. Weights, diffusion recipe and context are unchanged. Account separately for first-use compilation and warmed latency.
The following observer affects benchmarks only, not serving dispatch.
Emits descriptor, DiffusionGemma weighted-reduction, soft-conditioning and compiled-sampler dispatch counts from the first iteration, then requires unchanged disarmed counters. Does not select a route or alter serving. Use exclusive ownership, report first-use latency separately and exclude that iteration from warmed comparisons.
Bonsai performance qualification
These SDK controls default on for eligible paths of the unchanged schema-2
Ternary Bonsai 2 27B artifact. They are not a weight conversion, MTP capability
or deployment action. Source defaults apply to foreground and daemon processes.
Set any overrides before startup; these names are not in the LaunchAgent shell
environment passthrough. See libs/mlx-swift-lm/docs/bonsai2.md for the
artifact contract and qualification limits.
Earlier submission of compact recurrent carry during eligible packed text prefill; native arithmetic, deferred input fills, write-fault checks and engine retirement remain unchanged. Short/decode, media positions and captured windows retain existing scheduling. Other explicit spellings remain disabled.
Reuses the native FP16-to-FP32 scale/offset conversion for eligible 2-bit/group128/block1024 packed projections. Adds approximately 1.60 GB of retained constants for the selected pack; weights and native precision do not change. Descriptor/stream changes invalidate reuse; tracing falls back. Other explicit spellings remain disabled; the generic cache rollback remains effective.
See the matched performance report
for measured gains, tradeoffs and open gates. Neither control authorizes model
uploads, catalog changes, signing or production promotion.
The default reusable-buffer reader and bounded parallel hashing change full-file
reading and scheduling, not the verified
bytes or model arithmetic. Discovery remains hash-free. Fresh pre/post-load
verification and delayed identity publication retain their existing rules.
Variable
Values / type
Default
Read in
Effect
DARKBLOOM_EXPERIMENT_HASH_STREAM_FIRST
Unset or 1 enables; 0 and other explicit values disable
Bound independent file readers per invocation by file count and the selected limit, then combine every raw digest in the original sorted-key order. Any failed file prevents a successful aggregate.
Concurrent invocations each have their own worker bound; this is not a global
thread budget. A warm resident request that does not hash gets no direct benefit.
Leave both controls unset for the optimized default on every model. For the
original reading path and serial order, explicitly set
DARKBLOOM_EXPERIMENT_HASH_STREAM_FIRST=0 and DARKBLOOM_EXPERIMENT_HASH_WORKERS=1.
The existing variable names and invalid-value fallbacks remain compatible.
The controls do not enable caches, alter attestation policy or skip load checks.
Unset/empty uses the model default. Explicit affirmative values permit other models subject to capability/identity gates; resident payloads require the separate memory opt-in. Source enablement does not prove cache restoration.
Cadence of the local SSD stats line and typed per-store heartbeat observation; 0 omits the observation. Sample age still advances between ticks; see telemetry.
DARKBLOOM_PREFIX_CACHE_DISK_GB
GiB
Half the currently available space; 20 if space cannot be measured
Box-wide on-disk budget across all models, with no fixed default ceiling. A valid positive override is used verbatim. The separate 20 GiB free-space write reserve still applies.
Benchmark-only: use the normal persistent KEK path within an accepted test root. Fallback is still possible; the benchmark SPI defaults to requiring actual persistent mode. Not forwarded to LaunchAgents.
Attention staging byte/time caps. Complete checkpoints use the byte value as a payload-read cap; native destination plus bounded scratch is separately reserved before allocation, with no permanent RAM carve.
These overrides are read at slot construction in foreground/local and test
processes; neither is on the LaunchAgent environment allow-list. Both require
DARKBLOOM_PREFIX_CACHE_MEMORY=1 first; that opt-in and the global cache switch
are forwarded to the LaunchAgent. Backend/model/assistant
eligibility, measured publication limits, and the conservative reservation after
a slot shrink are described in
../architecture/prefix-cache.md#resident-tiers.
Provider cache enablement does not enable coordinator preference; the independent
EIGENINFERENCE_CACHE_ROUTING_MODE default remains off
(coordinator/registry/config.go, ReadConfig).
Reservation inside the existing slot KV grant; parsed values outside 0 < bytes < kvBytesCapacity disable the bank, malformed values use the default.
CBv2HybridPrefixCacheConfig defaults to maximumEntries = 32 and
maximumCheckpointsPerRequest = 2; these have no CLI environment overrides
(libs/mlx-swift-lm/Libraries/MLXLMCommon/ContinuousBatchingV2/Prefix/HybridPrefixCacheContract.swift).
Benchmark, harness and tests
Ordinary teacher-forced scoring is selected by CLI input, with no new environment
variable; its required input and backend are documented in the
CLI reference
(provider-swift/Sources/darkbloom/BenchmarkCommand.swift, teacherForcedOptionError).
Forces serial MTP verification in the Qwen canary.
Retired (parsed only to warn)
provider-swift/Sources/ProviderCore/Inference/Engine/Factory/EngineV2Config.swift recognises these and logs a warning; they have no effect: DARKBLOOM_ENGINE_V2, DARKBLOOM_ENGINE_V2_MODELS, DARKBLOOM_COMPILED_DECODE, DARKBLOOM_GEMMA_B1_FAST_PATH, DARKBLOOM_B1_GREEDY_FAST_PATH, DARKBLOOM_KV_GPTOSS_KERNEL, DARKBLOOM_ADAPTIVE_PREFILL_ALLOW_8192, DARKBLOOM_KV_CAPTURE_MAX_INFLIGHT, DARKBLOOM_PREFIX_CACHE_MIN_PERSIST_TOKENS. Six more names appear only in comments because mlx-swift-lm reads them, not the provider (DARKBLOOM_CBV2_ATTN_QUERY_BLOCK, DARKBLOOM_CBV2_PAGED_PTOK_TARGET, DARKBLOOM_CBV2_COMPILED, DARKBLOOM_GEMMA4_PREFILL_TAIL_ROWS, DARKBLOOM_GEMMA4_PREFILL_LAST_QUERY, DARKBLOOM_CBV2_MIXED_PREFILL_CAP); none is on the LaunchAgent allow-list, so they only apply under start --foreground.
Removed from the daemon's environment at start; reported as the env_scrubbed capability.
scripts/install.sh additionally reads COORD_URL (substituted by the coordinator when it serves /install.sh; required when the script is run from source), HOME (install root $HOME/.darkbloom), TMPDIR (enrollment-profile temp dir only) and the two code-signing requirement constants DARKBLOOM_DESIGNATED_REQUIREMENT and DARKBLOOM_FAN_HELPER_REQUIREMENT. See ../provider/installation.md.
These library controls apply to foreground processes and benchmark runs; they are not added to the provider LaunchAgent environment allow-list. Prefix reuse is independent of these changes.
Variable
Values
Default
Reader and effect
DARKBLOOM_GPTOSS_PREFILL_OUTPUT
full, intermediate, last, last-layer
last
libs/mlx-swift-lm/Libraries/MLXLLM/Models/GPTOSS+PrefillOutput.swift (GPTOSSPrefillOutputPolicy): skip unused intermediate vocabulary projections and project only the final hidden position. full restores full projections; intermediate preserves the final full-shape head; last-layer additionally narrows the last full-attention layer when its cache supports it.
DARKBLOOM_GPTOSS_FUSED_GATE_UP
0 disables; otherwise enabled
enabled for hidden/intermediate width 2880 and 32 experts
libs/mlx-swift/Source/MLX/ConstantArrayCastCache.swift (ConstantArrayCastCache): bounded reuse of unchanged BF16-to-FP32 constants; updates invalidate and transforms bypass reuse.
MLX_GPTOSS_MXFP4_DECODE_FAST_TAIL
1 enables, other explicit values disable
enabled only on physical applegpu_g16s
libs/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/quantized.cpp (gather_qmv): width-2880 MXFP4 gathered matrix-vector path with a masked 320-element tail. Exact shape/dtype gates retain the general fallback.
MLX_GPTOSS_MXFP4_PREFILL_TILE
m32n32k32; other values use legacy
legacy
libs/mlx-swift/Source/Cmlx/mlx/mlx/backend/metal/gptoss_mxfp4_policy.h (gptoss_mxfp4_prefill_tile): optional 32-row tile for matching sorted expert prefill shapes. Small workstation gains do not establish a universal default.
DARKBLOOM_GPTOSS_COMPILED_EXPERTS
1 enables
disabled
libs/mlx-swift-lm/Libraries/MLXLLM/Models/GPTOSS+CompiledExperts.swift (GPTOSSCompiledExpertsPolicy): compile single-token B=1/2/4 expert graphs for exact 20B shapes. The global MLX_COMPILED_DECODE=0 rollback still disables this path. Batch-dependent timing is mixed; weights remain live through weak updatable state.