Generalize native MTP beyond the Qwen3.5 singleton path (model coverage registry + batched MTP)

[Zigfreidish]

Problem

Native multi-token prediction (worker/runtime/native_mtp/) is a strong decode accelerator (~1.3–1.7× at typical accept rates per the cost model in batch_generator.py), but its current envelope is narrow:

• Model coverage: activation is gated on the Qwen3.5 config shape (qwen35_model.py, qwen35_vlm_model.py) with MTP head weights present. DeepSeek-V3-style MTP heads and other families that ship draft heads are not recognized.
• Batch shape: the patched GenerationBatch.next only engages MTP "when the batch holds exactly one MTP-capable sequence"; extend/filter deliberately drop MTP state when continuous batching reshapes the batch. Any concurrent load silently loses the speedup.
• Hardware gating: the known-limitation note in batch_generator.py (compute-bound M1/M2 single-stream decode can make MTP net-negative) is documented but not enforced — there is no device-aware auto-enable/disable policy, so operators must know the caveat themselves.

Proposed scope

• A declarative MTP capability registry (config-shape + weight-presence detection per family) instead of Qwen3.5-specific probing, starting with DeepSeek-V3-style num_nextn_predict_layers heads.
• Batched MTP: keep per-sequence MTP state across extend/filter for sequences that remain singleton-timeline-safe, or run the verify forward batched across MTP-capable sequences.
• Device-aware activation policy: use the existing Apple Silicon telemetry (see test_apple_silicon_telemetry.py) to auto-disable MTP where the 2-token verify forward is compute-bound, with an operator override and a receipt recording the decision.

Verification

• Greedy-identity contract tests per newly covered family (mirroring tests/test_mlx_lm_mtp_patch.py).
• Throughput probes on batch sizes 1/2/4 recording accept rate and net tokens/s vs baseline.
• Receipt fields for activation decision (enabled/compatible/weights_present/reason already exist in melix.native_mtp.* metadata — extend with the device policy outcome).

[Akane-CN]

Watch note — fail-closed speculative sidecar exposure

• Design principle: speculative accelerators should not become visible in CLI/help/effective config until the runtime can prove the target model, assistant sidecar, cache-sharing contract, and fixed-K fallback are all supported. Sidecar paths should be capability-gated just like native heads.
• Melix-specific goal: extend the native-MTP registry so requested_mtp resolves to accepted, legacy_only, or typed refusal reasons such as missing_head_weights, unvalidated_sidecar, unsupported_cache_sharing, or device_policy_disabled; do not advertise sidecar examples or profile defaults until those receipts pass.
• Measurable target: 100% of speculative serving starts emit a machine-readable MTP capability receipt; unsupported sidecar configs fail before model load; fixed-K bench mode is opt-in and recorded separately from production auto-K.
• Next executable slice: add a registry test fixture with one native-head checkpoint shape, one assistant-sidecar-shaped config that must be refused, and one legacy flag path that still resolves without exposing the new profile surface.

[Akane-CN]

Watch-note — native speculative capability should be routed through the same declarative acceleration contract as other decode modes.

• Design principle: an in-model draft head should be admitted by capability receipt + effective config, not by a legacy one-off flag or implicit model-family check.
• Melix-specific goal: make method=mtp resolve through the serving profile/config layer, reporting requested_method, resolved_method, model-family support, head-artifact presence, batch-shape eligibility, and hardware-admission decision before generation starts.
• Measurable target: for MTP-capable fixtures, profile startup emits one machine-readable receipt and the decode benchmark records acceptance/rollback counters; for non-capable fixtures, the request fails closed or downgrades with a typed unsupported reason and zero silent path changes.
• Next executable slice: add a config-parsing/receipt test that exercises speculative.method = mtp, then wire it into the existing native-MTP registry so the current Qwen3.5 singleton path becomes the first registered capability rather than a special case.

[Akane-CN]

Watch note — MTP/speculative configuration should be treated as one operator contract, not two parallel accelerators.

• Design principle: native draft-head acceleration should be exposed through the same effective speculative config that controls external draft models, with model-family capability detection and typed refusal reasons.
• Melix-specific goal: make native_mtp, draftModelID, numDraftTokens, batch eligibility, and hardware policy resolve into one receipt that says which accelerator was selected, why, and what disabled it when unavailable.
• Measurable target: for each supported family, produce a speculative_effective_config receipt containing {mode, source, family, head_count, batch_shape, hardware_gate, refusal_reason?}; add tests proving CLI/API config parity and no silent fallback when a native head exists but is ineligible.
• Next executable slice: extend the MTP capability registry work with an adapter that emits the same receipt schema as draft-model speculative decode, then add a single-family golden fixture before broadening coverage.

[Zigfreidish]

Implemented first slice via PR #2656 (squash merge 79e4ed13).

Completed:

• Added a native MTP capability registry for the existing Qwen native-head path.
• Added stable native-MTP receipt metadata for accepted, legacy-only, missing-weight, unsupported-model, assistant-sidecar, patch-failed, and invalid JSON payload paths.
• Routed text and VLM preload through the shared registry/preload helper.
• Kept assistant-sidecar-shaped configs fail-closed; no sidecar profile/default surface was advertised.
• Fixed review feedback by reading native-MTP JSON payloads as UTF-8 and treating malformed/decode-failed payloads as empty.

Verification on the merged PR:

• GitHub CI green, including python-tests and integration-tests.
• PR scoped performance report ok: 8 selected probes, 0 regressions, 0 verification failures.
• Local pre-commit passed after the review fix: make swift-test, make py-test, make integration-test, and scoped report ok.

Remaining #2602 work stays open: batched MTP execution, device-aware activation policy, additional model-family registry entries, and the unified speculative/effective-config contract.

[Akane-CN]

Watch note — native speculative depth and ownership contracts:

• Design principle: model-family coverage for native speculative decoding should be registry-driven and depth-aware, with cache rollback/acceptance state owned by the scheduler rather than hidden behind a single-family branch.
• Melix-specific goal: extend the existing MTP registry work so each supported family declares head count/depth, cache-extra shape, normalization repair needs, quantized-matmul compatibility, and whether batched rows can keep speculative state across extend/filter operations.
• Measurable target: at least three family fixtures cover depth-k acceptance, rejected-token rollback, singleton-vs-batched admission, and device policy; unsupported families emit a typed mtp_unsupported_reason instead of silently falling back.
• Next executable slice: add a NativeMTPCapability receipt object and tests around the current Qwen-family path first, then promote the second family only when the same receipt proves depth, weights, cache shape, and batch-state preservation.

[Zigfreidish]

Implemented second slice via PR #2671 (squash merge ebc5a20f).

Completed:

• Added a versioned NativeMTPCapabilityReceipt object for native-MTP capability decisions.
• Persisted the JSON receipt at melix.native_mtp.receipt_json while keeping the existing flat metadata keys.
• Covered accepted Qwen native-head and refused assistant-sidecar-shaped decisions with exact receipt assertions.
• Replied to and resolved the two Gemini review threads without code changes: explicit receipt mapping is intentional schema control, and _reason(..., override=...) remains the single normalization path.

Verification on the merged PR:

• Local focused native-MTP tests: 14 passed after each latest-main merge.
• Local diff checks: git diff --check and git diff origin/main...HEAD --check passed after each latest-main merge.
• Full local pre-commit before PR update: make swift-test, make py-test, make integration-test, scoped report ok.
• GitHub CI green after latest origin/main merge: python-tests 4m57s, integration-tests 29m55s, PR evidence pass.
• PR scoped performance report ok: 3 changed files, 0 selected probes, 0 regressions, 0 verification failures.

Remaining #2602 work stays open: batched MTP execution, device-aware activation policy, additional model-family registry entries, and the unified speculative/effective-config contract.

[Akane-CN]

Watch-note: native multi-token prediction needs an explicit feature-composition contract before it expands beyond the current singleton path.

• Design principle: acceleration features should declare shape, cache, and kernel compatibility before dispatch; decode-shaped multi-row requests must not silently enter prefill-only kernel routes.
• Melix-specific goal: add a compatibility matrix for native MTP + quantized active-KV + SDPA/prefill kernels, with typed fallbacks when cache positions, row shape, or kernel family are unsupported.
• Measurable target: MTP coverage fixtures include single-row decode, decode-shaped multi-row, chunked prefill, and quantized-KV sessions; unsupported combinations emit capability_receipt.unsupported_reason, while supported combinations report accepted kernel path and no full-cache dequantization on decode.
• Next executable slice: add capability probes around the current native-MTP registry, then gate quantized-KV and multi-row decode admission before widening model coverage.