Repository navigation
[Misc] Main2Main v0.33.0 - #20627
[Misc] Main2Main v0.33.0#20627
Conversation
f96f9b9 to
d026898
Compare
github-actions
Bot
added
documentation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request upgrades the vLLM compatibility baseline to v0.33.0, ensuring the Ascend backend remains synchronized with upstream developments. The changes encompass widespread updates to Dockerfile configurations, CI release tags, and internal architectural adaptations. Key refactors include updating KV-cache planning, attention dispatch, and speculative decoding logic to match the v0.33.0/main interfaces, while simultaneously cleaning up legacy code paths to maintain a streamlined codebase. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the vLLM Ascend integration to support v0.33.0, involving significant refactoring of KV-cache planning imports, model configuration updates (such as DeepseekV4), and the addition of priority-based preemption logic in schedulers. It also addresses various missing imports and type-checking issues across the codebase to ensure compatibility with the updated vLLM version. Both review comments provided identify actual missing imports that would cause runtime errors and should be addressed.
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
1 similar comment
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
3de0971 to
7434afe
Compare
|
/nightly all --vllm-ref v0.33.0
|
|
/nightly all --a3-560t --vllm-ref v0.33.0
|
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
7434afe to
2cc1a84
Compare
|
/nightly qwen3-30b-a3b-bf16-a2-performance,Kimi-K3-MRV2-A3 --vllm-ref v0.33.0
|
zhao-stack
changed the title
|
/rerun Rerun (failed jobs only):
|
2 similar comments
|
/rerun Rerun (failed jobs only):
|
|
/rerun Rerun (failed jobs only):
|
06f0ad4 to
c47b696
Compare
Audit every 0.32.0 guard against v0.33.0 (4ba787ec) and the fixed main pin (24420b7a). Remove obsolete branches and the upstream-fixed MM normalization backport; retain explicit 0.33/main ABI differences. Update Dockerfile defaults, CI release pin, documentation and matching tests. Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
Keep release-specific Kimi fused-gate coverage and parameter-rebinding assertions: v0.33 lacks the main-only own-storage behavior. Pass the builder to Qwen4 admission tests to match both supported upstream planner APIs. Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
The AMD Kimi attention inherited by Ascend already constructs KimiK3MergedQKVGateLinear in v0.33. Remove the obsolete release-only two-shard path so preprocessing peels and applies the gate, disables incompatible MLAPO, and preserves the KV shard boundary. Enable the fused-layout regressions for both supported versions. Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
Use the shared PCP input-buffer tuple contract and update newly merged DP sync fixtures to the v0.33 dataclass boundary. Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
vLLM v0.33 trims DP dummy draft requests while preserving the padded target InputBatch. Scope that effective count around drafting and use it for sequence/query metadata and initial DCP preparation. Preserve the target batch and restore the count on exit; reject invalid graph capacity. Local production-method CPU UT reproduced the 32-to-19 mismatch before the fix. Afterward, 50 tests passed against each supported upstream metadata implementation. Forward/NPU boundaries were mocked; full CPU imports and NPU E2E remain unverified. Test evidence stays outside this commit as requested. Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
Initialize the scoped draft metadata request count as int | None so mypy accepts restoring its absent value. This fixes the Python 3.10, 3.11 and 3.12 pre-commit assignment error without changing draft request-count behavior. Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
Record the tiling dependency and why replay metadata rebuilding currently needs the effective draft request count. Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
c47b696 to
72af94a
Compare
|
/nightly Kimi-K3-MRV2-A3 --vllm-ref v0.33.0
|
Provide num_reqs in the PCP synchronization fixture to match InputBatch's required fields after draft metadata request count alignment. Preserve all parameter combinations and assertions. Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
|
/rerun Rerun (failed jobs only):
|
### What this PR does / why we need it? Fix a mixed-precision weight-loading regression exposed by the vLLM 0.33 Kimi MLA gate fusion. Upstream AMD Kimi now packs `q_a_proj`, `kv_a_proj_with_mqa` and `g_proj` into `KimiK3MergedQKVGateLinear`. Ascend's ModelSlim mapping still describes only the first two shards. For quantized Q/KV with a FLOAT gate, the fused layer therefore allocates INT8 storage for all three shards. The real upstream weight loader silently casts BF16 gate weights into INT8, while the gate's quantization scale/offset shards have no checkpoint values. The shape fix in [d0e83d5](d0e83d5), merged with #20627, corrected output splitting but did not cover this allocation/loading mismatch. - Keep differently quantized MLA output gates separate from the replicated Q/KV projection. - Load standalone verifier and MTP gate parameters before the upstream fused-shard remapping, preserving TP slicing, scale loading and loaded-name tracking. - Validate all three quantization descriptions only for actual three-shard projections. Preserve all-FLOAT/uniform-quant fusion, nongated draft projections and existing KDA mappings. ### Does this PR introduce _any_ user-facing change? Yes. ModelSlim Kimi checkpoints with mixed MLA projection/gate precision retain the gate's checkpoint dtype and values instead of silently corrupting them. The public [Eco-Tech/Kimi-K3-w4a8 description](https://modelscope.cn/models/Eco-Tech/Kimi-K3-w4a8/resolve/master/quant_model_description.json) has W8A8_DYNAMIC Q/KV and FLOAT gate weights in MLA layer 11. The metadata response was last modified on 2026-08-28. Tiny synthetic weights reproduce that real precision layout; no model weights were downloaded. Related nightly evidence: - [Previous vLLM 0.32 case](https://github.com/vllm-project/vllm-ascend/actions/runs/42490093558/job/128919271821): GPQA 93.33%, completed. - [vLLM 0.33 case](https://github.com/vllm-project/vllm-ascend/actions/runs/42635278737/job/129224636495): hit the 120-minute limit while generating. Both report driver 25.5.2 and the same testcase configuration. The GPQA-window accepted/drafted totals fall from 91,417/410,732 (22.26%) to 20,455/1,335,936 (1.53%). This PR fixes a demonstrated code defect. The nightly's local checkpoint revision was not recorded, and hardware recovery of acceptance, accuracy and runtime has not been established. The defect is not claimed to be the sole cause of that timeout. ### How was this patch tested? - Real vLLM 0.33 projection construction and checkpoint loading on CPU: before the fix, BF16 gate value 0.25 silently becomes INT8 zero and gate scale/offset sentinels remain unfilled. The new regression fails on the original code. - Final related suite: **646 passed**, 14 warnings, using vLLM 0.33 / PyTorch 2.10.0+cpu. - Covers TP1, TP2/rank1, TP16/rank15, replicated Q/KV weight/scales, TP gate weight/scales, floating and quantized standalone gates, uniform fusion, absent/conflicting metadata, bare keys, loader kwargs, MTP checkpoint namespaces, nongated drafts, existing Kimi/DSpark/EPLB/SP and MLA behavior. - Relevant upstream projection/model/loader ASTs are identical between v0.33 `4ba787ec` and the supported main pin `24420b7a`. This is a source compatibility check, not a second runtime test lane. - Full `bash format.sh ci` and `git diff --check`: passed. - No NPU is available locally. Fixed-checkpoint NPU eager/capture, TP/SP, DSpark acceptance, GPQA and runtime regression remain required before merge. No nightly comment trigger has been sent. CPU suite command (Python 3.12, vLLM v0.33.0, PyTorch 2.10.0+cpu, repository NPU mocks): ```bash TORCH_DEVICE_BACKEND_AUTOLOAD=0 PYTHONDONTWRITEBYTECODE=1 PYTHONPATH="$PWD" \ python -m pytest -q \ tests/ut/models/test_kimi_k3_mla_quantization.py \ tests/ut/models/test_kimi_k3_mtp_gate_loading.py \ tests/ut/models/test_kimi_k3_adapter.py \ tests/ut/models/test_kimi_k3_dense_sp.py \ tests/ut/models/test_kimi_k3_dspark_weights.py \ tests/ut/models/test_kimi_k3_eplb.py \ tests/ut/models/test_kimi_k3_pp_sp.py \ tests/ut/quantization/configs/test_modelslim_config.py \ tests/ut/attention/test_mla_v1.py \ --disable-warnings --maxfail=1 ``` Release note: `release_notes/fragments/21387.yaml`. ### Checklist - [x] **Boundary / edge-case coverage** - [x] **Regression test** - [x] **Error / failure path coverage** - [x] **Meaningful & reliable tests** - [ ] **Performance impact**: NPU validation remains pending. - vLLM main: vllm-project/vllm@24420b7 --------- Signed-off-by: zhao-stack <80399320+zhao-stack@users.noreply.github.com>
What this PR does / why we need it?
Upgrade vLLM release compatibility and Dockerfile defaults to v0.33.0 (
4ba787ec9959e566d3b587a796dff07673324da5). The verified vLLM main pin remains24420b7a341af37a9bc3186e6fc6062512e0aa58. The separate main-pin upgrade is in #21005.The five signed-off release commits are rebased onto Ascend main
7ead678c715ccf292d19f6b2f2ff41c0fe0e4052:19e32a64e: release version guards, Dockerfile defaults, metadata and documentation.3708328d6: v0.33 CPU expectations and KV planner calls.d0e83d582: Kimi fused output-gate compatibility.0362753e7: v0.33 DSpark sampling fixture.70c2b2be9: PCP tuple contract and DP sync fixtures.The clean release history avoids replaying reverted main-pin changes. After the previous rebase, upstream #20361 changed adaptive verification and CI's live-main rebase conflicted in three files before tests. This rebase keeps upstream's capability-driven allocator, its legacy/Triton tests, and its 3/4-field allocation handling with admitted-draft propagation. The obsolete overlapping release hunks were removed; the other four release patches remain identical. No unrelated upstream feature fix, main-pin upgrade or benchmark threshold change is included.
Does this PR introduce any user-facing change?
The supported release baseline and Dockerfile defaults become vLLM v0.33.0. The verified main pin is unchanged. Release note:
release_notes/fragments/20627.yaml.How was this patch tested?
Current head:
70c2b2be95cd66808c88e46aae0bf1c98f951de0. E2E42000109512 has passed pre-commit, test selection and both CPU lanes; each CPU lane reports 16516 passed / 109 skipped, including all three allocator compatibility tests and successful rebase. Attempt 2 is now terminal success: 44/44 hardware shards passed, ci-gate passed, and 50 jobs succeeded / 8 skipped / 0 failed overall. Skipped jobs are not test passes.Python AST parsing, Ruff lint and Ruff format checks pass for all 104 changed Python files.
git diff --checkpasses and CI-style rebase onto the current base succeeds as a no-op.Range-diff confirms the four subsequent release patches are identical. The first patch differs only by removal of the adaptive-verification hunks superseded by upstream. The allocator patch and its tests exactly match current upstream.
Existing adaptive-verification tests cover legacy allocator wrapping and preservation of the Triton allocator. All three allocator tests passed in both current-head CPU lanes; local Windows lacks the Torch runtime.
Full
bash format.sh ciwas attempted with bounded local initialization and stopped at actionlint setup; the task's own formatter process trees were cleaned. A complete local lint pass is not claimed.Previous head
5f9371a39, E2E41996832071: pre-commit passed; both CPU lanes and test selection failed during rebase, before tests, with the three adaptive-verification conflicts repaired here. Those failures are not runtime test failures.Historical head
6e4535561: CPU lanes each passed 16398 tests / 109 skips, and 43/44 hardware shards passed. Release A2 part7 failed before tests on the PyTorch ninja index HTTP503; those old-head results do not validate this head.Attempt 1 main24420 A3 job127162106257 failed the Qwen DSpark GDN fixed-versus-plain comparison (prompt2/token19, logprob difference 0.1472 > 0.1378). On unchanged published head, the targeted attempt 2 job127250533233 passed the same file (2 tests), the full shard (10 passed / 1 skipped), and ci-gate127271942785 passed. No tolerance or source changes were made between attempts. The rerun was triggered by
vllm-ascend-ci; this monitor only observed it. Runtime vLLM remains0.1.dev1+g24420b7a3.empty; Ascend is0.19.1rc2.dev3366+g3603d176eversus attempt 1...+g07b5dafcd, because CI rebases onto live main and produces a local detached commit. Exact rebased full SHA/tree, image digest and weight revision are not established. The first numerical failure's root cause remains unproven; rerun success does not by itself prove fluctuation or exclude every migration association. The release counterpart also passed the same two tests.Follow-up fixes are limited to v0.33 migration regressions and rebase/restore operations. Other-code failures will be reported separately. Skipped or unexecuted coverage is not counted as passing.
Historical A2 performance, four-node Kimi DSpark, A5 GLM dtype and A5 DSV4 layout risks remain outside ordinary-CI closure. This repair did not post a new nightly comment or restart local hardware validation.
vLLM main: vllm-project/vllm@24420b7