Skip to content

[Feature][HiSparse] Import P KV into Ascend Decode Host Source - #2

Closed
LifeOfCoffee wants to merge 2 commits into
Esir98:fix/hisparse-ascendstore-colocatedfrom
LifeOfCoffee:codex/hisparse-pd
Closed

LifeOfCoffee wants to merge 2 commits into
Esir98:fix/hisparse-ascendstore-colocatedfrom
LifeOfCoffee:codex/hisparse-pd

Conversation

@LifeOfCoffee

Copy link
Copy Markdown

What this PR does / why we need it?

Adds HiSparsePDConnector for asymmetric P/D serving: Prefill uses ordinary KV plus AscendStore/Memcache; Decode uses HiSparse and pulls KV directly from Prefill. The connector reuses Ascend Mooncake/HIXL transport without adding a NIXL dependency.

The Decode import writes main MLA KV into Host Source and Indexer KV into NPU. HMA's Host Source is already ACL-registered, so Mooncake transfers through a separate pinned Host staging region. Split Prefill NoPE/RoPE components are repacked by token into HiSparse Source rows before Core completes the Host import and restores the last Resident page.

This is stacked on the existing colocated pooling branch (Esir98/vllm-ascend@5ebaa173), itself based on vllm-project#20926.

Does this PR introduce any user-facing change?

Yes. It adds a connector and documents the P/D configuration. A numbered release note fragment will be added to this draft PR.

How was this patch tested?

  • ruff check, ruff format --check, Python compile check and git diff --check passed.
  • On Ascend 950, GLM 5.2 10-layer BF16, P TP2 on cards 4–5 and D TP2 on cards 6–7: a 4-token one-page prefix and a 221-token two-page prefix both transferred and produced byte-for-byte identical Decode output to the D-local baseline. D transfer logs show both TP ranks.
  • Three focused HiSparse PD unit tests passed on that NPU container.
  • P's repeated 221-token request incremented AscendStore/Memcache GET keys to 2. D had no pool connector.
  • The remote container used the [Feature][HiSparse] Add Ascend backend foundation vllm-project/vllm-ascend#20926 wheel with updated Python Mooncake modules, not a full wheel built from this branch. Exact-branch build/CI and broader model coverage remain pending. bash format.sh ci could not run on the Windows host because Bash is unavailable.
  • The separate D Host staging region currently uses capacity equal to the Host Source pool (4 GiB per D rank in this run). Memory cost and performance under concurrency need review before merge.

Checklist

Testing

  • Boundary / edge-case coverage: one-page and two-page imports.
  • Regression test: token-wise MLA repacking reproduces the initial wrong-output failure.
  • Meaningful & reliable tests: unit and real P/D output comparisons.
  • Performance impact: memory and concurrency benchmarks remain pending.

Add a dedicated HiSparse PD connector that reuses Mooncake transfer while routing main KV through Host Source and Indexer KV to the NPU. Repack split MLA components by token before completing the Host import.

Tested GLM 5.2 10-layer TP2 P and TP2 D on Ascend 950 with one-page and two-page prefix continuation; both matched local Decode output. The remote test used a mixed Python installation, so an exact branch build remains for CI.

Signed-off-by: LifeOfCoffee <lcyzhyx@163.com>
Signed-off-by: LifeOfCoffee <lcyzhyx@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant