follow-up: vLLM Kimi-K3 W4A8 MegaMoE (deep_gemm) data lane — unblocked on post-merge vllm main

[tianhaox]

Context

Kimi-K3's checkpoint (compressed-tensors mxfp4-pack + SiTU) serves two vLLM MoE paths:

1. Single-node / TP: MarlinExperts weight-only W4A16 (CUTLASS W4A4's activation whitelist has no SITU) — collected and shipped in feat: Kimi-K3 support — hybrid KDA+MLA model, DSPARK, multi-arch sglang + vllm perf data (8 systems) #1435 on all eight systems, kernel lane w4a16_mxfp4.
2. Recommended DEP scale-out (--moe-backend deep_gemm_mega_moe): DeepGEMM MegaMoE, fp8 activations × fp4 weights = W4A8 semantics. This path was unrunnable on the pinned preview image (vllm/vllm-openai@sha256:e90e2603, 0.1.dev19262 — no deep_gemm package), so feat: Kimi-K3 support — hybrid KDA+MLA model, DSPARK, multi-arch sglang + vllm perf data (8 systems) #1435 carries no vLLM W4A8 rows; W4A8 scope is covered by the sglang trtllm-gen lane only.

What changed upstream (2026-08-02 research)

• K3 merged to vllm main Jul 29–30 ([Model] Add Kimi K3 support: model files and kernels [1/N] vllm-project/vllm#50089 + [New model] Kimi K3 vllm-project/vllm#50000); no tagged release contains it yet (v0.26.1rc0 was cut pre-merge; tracking issue [Model Support] Kimi K3 Tracking Issue vllm-project/vllm#50001).
• The blocker is gone on main: DeepGEMM is vendored into the official image (vllm.third_party.deep_gemm, [Build] Build bundled DeepGEMM _C per-Python so the wheel imports on every CPython vllm-project/vllm#41516) and the K3 merge pinned a SITU-capable vllm-project/DeepGEMM fork (fp8_fp4_mega_moe kernels; ep_gather fix [Kimi K3 Bug] Fix deepgemm support for kimi k3 vllm-project/vllm#50458). deep_gemm_mega_moe should run on nightly-* images (Jul 31 onward).
• CUTLASS W4A4 SITU whitelist is still not extended → the single-node W4A16 lane label stays serving-true; nothing to re-collect there.
• New on main: FlashInfer trtllm-gen mxfp4×mxfp8 SITU experts exist behind a FlashInfer capability check, but whether the compressed-tensors scheme can route into them is undetermined.

Follow-up work (the actual scope — large, hence this issue)

• Pick and pin a post-merge image (nightly or the first release containing K3). A pin bump triggers the standing full kda dispatch re-audit (merged-main routing may differ from the preview branch: FlashKDA/fused_kda_decode availability per SM, backend selector changes).
• vLLM MegaMoE module lane collector (peer of the sglang MegaMoE module lane): needs 8-GPU Blackwell + --enable-expert-parallel; verify KimiK3MegaMoEExperts construction against serving (SiTU, latent MoE, grouped top-k).
• Collect W4A8 rows on Blackwell systems; decide version-dir/labeling for nightly builds (setuptools-scm dev versions).
• Re-run the K3 vllm serving-truth audit on the new pin (Marlin fallback vs CUTLASS vs trtllm-gen routing for the compressed-tensors path).
• Support-matrix / SDK: decide whether the vllm K3 resolver should query w4a8 on Blackwell once data exists (mirroring resolve_kimi_k3_moe_arch_mode for sglang).

References

• Data/audit record: docs/perf_database/kimi-k3-blackwell-bringup-ledger.md on the feat: Kimi-K3 support — hybrid KDA+MLA model, DSPARK, multi-arch sglang + vllm perf data (8 systems) #1435 branch (sections "vLLM K3 serving-truth audit" and "vLLM upstream W4A8 status check — 2026-08-02").
• Owner decisions 2026-08-01/02: keep pin e90e2603 for feat: Kimi-K3 support — hybrid KDA+MLA model, DSPARK, multi-arch sglang + vllm perf data (8 systems) #1435; W4A8 = this follow-up.