[RFC] Delta weight sync requirements

[athreesh]

RFC: Delta Weight Sync Requirements

Decision

Implement delta-compressed weight updates in ModelExpress as a first-class source type for async RL and disaggregated rollout fleets.

Fast Review Path

• Read Requirements to approve if you want to approve or reject the product boundary.
• Read Acceptance criteria if you want to check what P0 must prove.
• Read Open questions if you want to discuss ownership, runtime scope, or transport choice.

Summary

• MX should support delta-compressed weight updates as a first-class source type.
• A trainer publishes full checkpoint anchors and small deltas between anchors.
• A rollout worker fetches, rebuilds, checks, and stages the next version before it pauses inference.
• The worker then applies the staged version through the serving runtime and resumes.
• MX owns the control plane: version chain, readiness, tensor identity, target demand, fallback, and metrics.
• MX does not need to own one fixed delta format or one fixed runtime apply path in P0.

Why

Async RL needs rollout workers to stay near the trainer policy. Full checkpoint reloads are too slow and expensive when the model is large or the fleets are not colocated.

Public systems show the same pattern. Cursor and Fireworks describe frequent policy updates with less data movement. Cognition describes compressed weight diffs, cluster-local control, background fetch, and a short apply pause. Slime gives an open-source version with shared storage, per-tensor deltas, local checkpoint patching, integrity checks, and reload through the normal runtime disk path.

MX should make this pattern reusable across frameworks. The MX value is not the compression trick by itself. The MX value is the safe update contract around versions, tensors, target demand, readiness, fallback, and observability.

References

• Slime: Delta Weight Sync example
• Fireworks: Cursor Composer 2 + Fireworks AI
• Fireworks: Frontier RL Is Cheaper Than You Think
• Cognition: SWE-1.7
• Cursor: Improving Composer through real-time RL
• Sequoia: How Cursor Trained Composer on Fireworks

Requirements to Approve

• R1. Track the version chain. MX tracks full checkpoint anchors and delta updates between anchors. Approve if MX must know checkpoint lineage, not only the latest available weights.
• R2. Validate delta metadata. MX stores enough metadata to check model identity, base version, target version, tensor identity, layout compatibility, checksums, and rebuild requirements. Approve if MX must stop wrong-version or wrong-layout updates before the runtime applies them.
• R3. Reject unsafe bases. MX rejects a delta when the worker is not on the required base version, unless MX can replay from an earlier anchor. Approve if silent drift is not acceptable.
• R4. Stage before pause. MX exposes a fetch/process/stage phase. Workers download deltas, rebuild required weights or shards, and check them while they continue to serve. Approve if the main value is short rollout downtime.
• R5. Apply during a short pause. MX exposes an apply/resume phase. The runtime installs only weights that are already staged. Approve if transfer and rebuild work must stay outside the runtime mutation window.
• R6. Capture target demand. MX lets a worker declare the tensors, ranges, TP/PP/EP shards, or local experts that it needs for a target version. Approve if MX must avoid moving data that a worker does not need.
• R7. Support normal runtime reload paths. MX supports a path where deltas patch a host-local checkpoint and the runtime reloads from disk. Approve if P0 must support the Slime/SGLang-style path without a new GPU-native delta apply API.
• R8. Make partial update rules explicit. MX records which tensors can update alone, which tensors must update together, which tensors can be skipped, and which tensors need full reload or fallback. Approve if correctness rules must be visible in MX instead of hidden in framework-specific code.
• R9. Use shared storage as a durable source. MX supports shared storage for delta payloads and metadata. This includes object-store-backed filesystems that need explicit visibility or sync hooks. Approve if the use case includes wide-area async RL.
• R10. Support local fanout. MX supports cluster-local fanout after one controller or worker fetches a delta. Fanout can use local disk, node-local cache, or tree broadcast. Approve if MX must reduce repeated shared-storage downloads and trainer egress.
• R11. Check correctness. MX checks reconstructed weights and confirms the final runtime version. Approve if READY must mean that the worker is on the intended version.
• R12. Recover from normal failures. MX handles missed versions, bad deltas, stale workers, and lost local checkpoint state. Approve if restarts and delayed workers are normal in async RL.
• R13. Report useful metrics. MX reports compressed bytes, reconstructed bytes, skipped bytes, fallback bytes, full-checkpoint bytes, trainer egress bytes, shared-storage egress bytes, local fanout bytes, and phase timing. Approve if we need fair comparisons against NIXL, NCCL M2N, full checkpoint reload, and framework-native broadcast.

P0 Acceptance Criteria

• A1. Publish source. A trainer or publisher registers one full checkpoint anchor and one compressed delta update in MX. This proves that MX can ingest delta updates as a source type.
• A2. Stage before pause. A rollout worker asks MX for the next version, fetches the required delta payload, rebuilds the needed weights, checks them, and reports staged readiness before it pauses inference. This proves that most work happens outside the serving pause.
• A3. Apply through a runtime. A rollout worker pauses, applies the staged version through a framework-native update hook, verifies the final version, and resumes. This proves that the flow works at a real serving boundary.
• A4. Support a disk reload path. A worker can patch a host-local checkpoint and reload through a normal runtime disk update API. This proves that MX can support the open-source Slime/SGLang shape in P0.
• A5. Refuse unsafe updates. MX refuses to apply a delta if the base version, tensor metadata, layout tags, or checksum checks do not match. This proves that MX enforces the safety contract before runtime mutation.
• A6. Recover a stale worker. MX can recover a worker by replaying deltas from the latest checkpoint anchor or by routing it to a full checkpoint fallback. This proves that the system handles missed versions, restarts, and stale workers.
• A7. Split timing. MX reports manifest_publish_ms, fetch_ms, process_ms, stage_ms, pause_ms, apply_ms, and resume_ms. This proves that reviewers can separate transfer cost, rebuild cost, and inference downtime.
• A8. Match a reference. We can compare a delta-rebuilt version with a full checkpoint reference and get byte-equivalent or explicitly tolerance-equivalent weights. This proves that delta chains do not create silent corruption or drift.

Open Questions

• What does MX own, and what does the framework own?
• Is P0 only a metadata and control-plane feature, or does MX include a reference fetch/rebuild/stage client?
• Should P0 start with the disk/local-checkpoint path because it matches Slime, or should it start with a runtime-native path for vLLM or SGLang?
• How does MX choose between delta artifacts, NIXL, NCCL M2N, full checkpoint fallback, and framework-native broadcast?
• Which P0 metric matters most: trainer egress, rollout pause time, policy staleness, or end-to-end training throughput?
• Which partial update cases are required in P0: MoE experts, fused layers, quantized inference formats, FP8 metadata, or only dense tensors?
• What must vLLM, SGLang, Megatron, NeMo RL, and TRT-LLM expose so this can be robust?
• What correctness bar should MX enforce: byte-exact weights, tolerance-equivalent weights, per-tensor checksums, or end-to-end version attestation?

Product Stance

Delta weight sync should not be a one-off loader path in MX. It should be a first-class source type that reuses MX strengths: tensor identity, target demand, partial update rules, version fences, readiness, fallback policy, and metrics.