[zhenwenqi2024]

What this PR does / why we need it?

Add the Ascend backend for the merged HiSparse interfaces. Core owns HMA allocation, Coordinator admission/eviction, request lifetimes and completion. Ascend supplies native residency resolution, mapped-Host transfers, shared-slab layout adaptation and calls to the existing SFA operations. Attention core arithmetic is unchanged; no unmerged vLLM Core patch is required.

Supported scope: Model Runner V2, Ascend 950 with a SIMT-capable CANN 9.2 toolchain, BF16 KV or native E4M3FN C8 KV with FP32 scales. Main and Indexer KV formats are independently selectable. INT4 KV, TurboQuant, context parallelism and external P/D transport are outside the current qualification.

Implementation and necessity

• Reuse Core cache specs, HMA, Coordinator and lifecycle hooks. The existing allocation adapter in hisparse/host_pool.py covers the missing private Host-allocation hook and retains Core cleanup.
• Build the AscendC resolver through the repository CMake/extension path. Preserve MTP query unions and emit Attention-ready C8 indices plus compact refill plans. There is no Triton resolver fallback.
• Retain the live Triton transfer/layout/invalidation operations required by Host/Resident/Hot storage, Prefill and mixed batches. They are not alternative residency resolvers.
• Reuse merged Core #63801 group prefetch, including MTP verification. The Ascend Prefill bridge retains current request-row mapping, bounded lookahead and staging ownership.
• Reuse one byte-copy Resident gather for initial Prefill staging and the lookahead patch, preserving their different invalid-row contracts and the existing stream/completion ownership.
• When the existing SFA selection chooses native C8 Prolog V3, write directly to Core's write_target() cache by default. Pad a shortened valid slot prefix with -1; reuse the existing device producer for replay padding. Remove the runner opt-in binding and Prolog staging/scatter branch. Other preprocessing operations and Attention arithmetic are unchanged.
• Retain the reviewed grouped invalidation, slot-dtype matching and row-DMA preparation, including completion and overwrite fences.
• Share RoPE metadata between compatible ordinary SFA and Indexer builders within one metadata build. Remove the HiSparse opt-in and execution-phase/topology guard chain from the caller and common utility; CP forwards the same context. Changes to position/table/destination owners and tracked versions invalidate reuse. Custom providers, callbacks and unrelated builders are write boundaries; draft steps retain independent storage.
• Keep Host allocation/registration in hisparse/host_pool.py. A complete Target-only Host writeback can overlap drafting when the bound HMA layer/page ownership and draft group isolate the source and destination rows. Keep the output durability dependency and join at Core's draft finalizer; incomplete plans keep Core's original wait. This uses the existing DMA and does not snapshot KV payloads.

Independent component evidence includes removal of 40 captured Cast tasks by slot-dtype matching, with 38%–50% lower component Host submission; the eager 8192-written-row invalidation path also has recorded device and same-work model Prefill benefit. These retain their original source/configuration scope. The model comparison below qualifies the combined implementation, not a separate speedup for each helper.

The file-by-file review removed unread buffers, duplicate storage retention and DMA cases, unchanged Graph replays, duplicated CPU token-combination logic and per-batch diagnostic state. The logits-index cleanup removes one dedicated E2E test file and updates three existing files. The budget follow-up below changes five already-present paths and adds no benchmark framework or native kernel.

Memory-budget alignment and retained owners

The budget path reuses Core's measured result. Follow-up a11b1b9d1 moves workspace discovery and initialization into the HiSparse backend; Model Runner V2 now exactly matches the PR target-base file.

• Preserve Core's already measured memory_profiling.non_kv_cache_memory for the HiSparse KV budget and startup suggestion. Do not reproduce its formula. Ordinary-model accounting is unchanged.
• The Worker calls backend preparation once before both automatic profiling and explicit-KV-size branches. The backend finds bound Target/draft runtimes and uses Core's WorkspaceManager.get_persistent_resource once per actual index group. Core's startup-snapshot total-consumption measurement includes these retained allocations, without a second budget formula. Repeated preparation keeps the same pending storage; real binding consumes it. Follower sharing and standalone binding remain supported.
• Prepare per-binding C8 length tensors without cloning. Allocate normal tensors even when profiling runs under inference mode, so later ordinary-mode initialization is legal. Rebinding retains independent length storage for old Graph owners.
• Keep the existing HMA-routing bypass of Ascend's private-per-layer multi-group scaling; applying that scaling to HMA would reduce the upstream budget again.

This explicitly covers resolver scratch and C8 length storage. It does not establish accounting for every late snapshot/retired-pointer allocation or claim a reproduced OOM fix.

Does this PR introduce any user-facing change?

Yes: experimental Ascend HiSparse support. Configuration, build requirements and support limits are in docs/source/user_guide/feature_guide/hisparse.md; release note: release_notes/fragments/20926.yaml.

How was this patch tested?

Regression and static checks

Copy-stream cleanup (b22dbf6): HiSparse uses Core copy streams directly. On one Ascend 950 with torch-npu 2.10.0.post7 and Core39, both wrapped and plain torch.Stream passed the same two TopK 2048 cross-stream Graph cases. The committed removal passed all six resolver/refill and workspace-rebind Graph cases. Ruff, formatting and diff checks passed. Full-format CI stopped during actionlint setup.

Backend interface cleanup (source 3f48a57)

Use Core's grouped pointer-table ABI for written-slot invalidation, including a single group, and update the existing hot-capacity Graph test to exercise that ABI. Install the existing row-DMA and recycled-slot adapters directly, removing the whole-file source hash allowlists and optional recycled-adapter installation branch. 160 source-isolated CPU dispatch comparisons, 36 real-CPU descriptor comparisons against Core39 and four installation checks passed. Focused Ruff, formatting and diff checks passed. The required full bash format.sh ci attempt stopped after 45 seconds during actionlint environment installation. NPU eager/Graph and whole-model tests have not been rerun for this cleanup; earlier results retain their recorded source scope.

At frozen public head 8d044bc1d0a7d5466f2e04af317a7dfcdfdda950, 28 actual-package CPU cases and 42 actual NPU cases passed, with no failures or skips, using merged Core39, Python 3.12 / CANN 9.2.1 and unchanged native9bd. CPU tests used the original conftest and real package imports, checked module origins, and forbade NPU initialization. The NPU set comprised 12 BF16 sparse-Attention reference cases, 8 C8 Prolog cases, 2 profiled-workspace/rebind cases, 16 Resident-payload cases and 4 changing-extent Resident patch cases. Four zero-row Resident cases returned normally without Graph capture; nonempty eager/Graph cases retain their actual test scope. Owned processes were reaped and leases released after both suites. These results supersede the earlier pending component status for these selected tests. A subsequent whole-model startup at this predecessor exposed the capture-interface issue corrected below; no whole-model result is claimed for this predecessor.

Capture-interface correction aa7f5dd79 preserves Core's build_for_cudagraph_capture(common_attn_metadata) signature. Shared RoPE context remains on the runtime build() interface. The existing regression test now inherits the real Core/Ascend capture entry points instead of overriding them with a permissive mock; the revised 28 actual-package CPU cases passed with no skips and NPU initialization forbidden. Actual TP8 model startup then completed capture on all eight ranks, activated Target/draft replay, passed functional requests and the unscored primer, and began the OFF formal rounds. The transfer/Attention implementations and native assets used by the earlier 42 NPU tests are unchanged; those tests retain their 8d044bc1d source scope and were not rerun at this correction. The corrected-source ON/OFF N2 completed as recorded below. Focused Ruff, formatting, syntax and diff checks passed; full-format CI again timed out installing actionlint after 55 seconds, so full CI is not claimed.

Resident-gather follow-up f414e7a41 reuses the existing byte-copy kernel for initial Prefill Resident staging and the lookahead Resident patch. Initial staging preserves Core's negative-ID-to-row0 convention; the patch retains staged bytes at invalid IDs. The common wrapper handles flat/paged staging, int32/int64 row IDs, byte-compatible payload views and empty inputs, without an intermediate gather allocation. 96 actual Triton-interpreter cases passed against an independent whole-backing-byte oracle, including page gaps, storage offsets and tails. The wrapper's NPU-device guard was adapted for CPU; NPU initialization was forbidden. The 64 nonempty interpreter cases each submitted one kernel, and empty inputs submitted none. These are not hardware kernel counts or performance measurements. Existing one-card eager/Graph tests were extended, not executed. Focused Ruff, format, syntax and diff checks passed; real NPU eager/Graph, transfer performance and whole-model validation remain pending. Full-format CI again timed out installing actionlint after 55 seconds; full CI is not claimed.

Intermediate-reuse cleanup 52e6d7879 changes one existing adapter file: reuse the BF16 validity mask/current valid lengths and the staged Prefill nonnegative indices. Ten source-isolated CPU comparisons passed, with identical native Attention input values, physical Prefill indices and results. CPU ATen dispatch counts decrease by three per BF16 chunk and one per staged Prefill mapping; these are not measured NPU kernel counts or TPOT gains, and the native-C8 Decode path is unchanged. Native Attention and payload transfer were contract doubles in these checks. No new test framework or kernel was added. Focused Ruff, formatting and diff checks pass; full-format CI again timed out installing actionlint after 55 seconds. Final-head NPU eager/Graph and whole-model validation remain pending.

RoPE reuse follow-up 62d66e65e: 28 source-isolated CPU contracts passed, executing the actual metadata dispatcher and RoPE lookup/cache functions with real CPU tensors and recording builder families. Cases cover ordinary/inference tensors, eager/capture dispatch, Prefill/dummy inputs, callback/provider barriers, cross-build position changes, and table/destination mutation or replacement. Focused Ruff, format, syntax and diff checks passed. Full pytest collection is blocked by the absent local vllm dependency; full-format CI timed out after 55 seconds installing actionlint. These are not actual-package, NPU eager/Graph, CP or whole-model results. Final-head correctness and performance qualification remain pending, and no new model speedup is claimed.

Workspace cleanup a11b1b9d1: ten existing CPU parameter cases passed through source-isolated imports, covering group isolation, follower sharing, repeated preparation, binding reuse, ordinary tensors under inference-mode startup and C8 length ownership across rebindings. The compiled native-capacity query was mocked; this is not full-package or NPU qualification. The unmodified Core memory_profiling() function, executed with controlled snapshots, includes workspace allocated before profiling exactly once, including repeat preparation. Its implementation is identical at Core39 and the CI footer pin. Worker branch ordering and the byte-identical target-base Model Runner file were checked. Focused Ruff, formatting, syntax and diff checks passed; full CI again timed out during actionlint environment installation. Final-head NPU/model validation remains pending.

The preceding cleanup fd388b215 restores direct public Core token combination and Core mirror completion, and makes native C8 Prolog write to the bound final cache without runner opt-in. The existing Prolog E2E case now uses real Core cache binding/write-target methods and covers int32/int64 slots, shortened/empty effective prefixes, strided destination guards, changing -1 padding and eager/Graph execution. The direct-write cases are included in the a11 device requalification below.

Local source-isolated CPU contracts passed eight payload/mapping cases plus an int64 boundary check, using actual Core binding/write-target and Ascend methods with a contract-only parent byte writer. This does not execute native Prolog arithmetic or qualify full-package imports/NPU behavior. Focused Ruff, format, syntax, Markdown lint and diff checks pass. Local full-package import lacks vllm; full-format installation again hit the 55-second actionlint limit. These local CPU checks do not add NPU, model or performance qualification. The measurements below retain their original source scope.

At frozen source a11b1b9d1d1f2415e7e5953137bcae2202c240cb, actual one-card NPU regression passed 10 cases with no skips: the eight C8 Prolog eager/Graph cases above and two profiled-workspace rebind/Graph-owner cases. These used real installed packages, merged Core 39dbf5183, Python 3.12 / CANN 9.2.1 and unchanged native SHA256 9bd146e9ebb2b1ae830d32915e46cd41855b7b04933d556d6406ef33ebe20015. Owned test processes were reaped and the lease released. These source-scoped component checks do not qualify the later RoPE/intermediate-reuse follow-ups or establish new whole-model correctness or performance.

At production source c366b41c33551a3885fe9c4d33af203cae48b5dd, using the canonical test-file contents from test-only commit 6bab22a29:

• Foundation CPU pytest: 38 passed, with real installed Core/Ascend packages, original conftest and NPU initialization forbidden. Existing geometry cases cover standalone and profiled owners rather than duplicating setup.
• Actual one-card NPU pytest: 2 passed, no skips, covering inference-mode preparation, repeated preparation, ordinary-mode binding and two captured length owners across three bindings.
• The canonical files were executed as external test overlays through the existing CPU/NPU flows. Production/native bytes in public 6bab22a29 are identical to the measured c366b41c3.
• Existing Host-DMA, C8 scatter and Resident-payload Graph qualification passed 14 cases at 0271d60db; the relevant Host/transfer bodies are unchanged by the budget follow-up.
• Focused Ruff, formatting, syntax and git diff --check passed. The required final bash format.sh ci attempt remained in actionlint environment installation at its 55-second bound; full CI is not claimed.

Repository test entry points:

pytest -q tests/ut/distributed/kv_transfer/test_hisparse_foundation_cpu.py
pytest -q tests/e2e/pull_request/one_card/test_hisparse_prefill_staging.py::test_profiled_workspace_rebind_preserves_prior_length_graphs
pytest -q tests/e2e/pull_request/one_card/test_hisparse_kv_preprocess.py::test_c8_prolog_keeps_parent_outputs_and_hisparse_destination

Direct Target Host writeback overlap (production source 1f3b0d4)

The actual production diff has two Ascend Python files; signed follow-up f4a1dbb46 adds only two cases to the existing one-card test file. Both production and native bytes at that head match the model-tested 1f3b0d42. Core, Attention arithmetic, resolver, refill, and recycled invalidation are unchanged.

Actual A5 / Python 3.12 / CANN 9.2.1 with merged Core 39dbf5183 and unchanged native9bd, same host per workload, OFF then ON, each arm one unscored primer and two 32K input / 8K output rounds. The model and graph configuration match the GLM-5.2 TP8/MTP3/NZ0 cohort described below.

Workload	OFF TPOT (ms)	ON TPOT (ms)	Per-round increase	N2 mean increase
C4, zero reported miss	12.953 / 12.960	13.226 / 13.254	2.11% / 2.27%	2.19%
C8, reported miss	15.953 / 15.956	16.950 / 16.914	6.24% / 6.01%	6.13%

Both pairs match full input/output token IDs and total/per-position MTP, with zero preemptions. C4 had present, settled zero software miss/H2D-estimate deltas and met the 3% TPOT target in both rounds. C8 steady Decode throughput was 602.28 OFF / 565.96 ON tok/s. C8 request Prefill means were 9.980 s OFF / 10.245 s ON (+2.65%); those histograms include scheduling. All eight ranks logged using the direct overlap path and queuing the wait after draft in the C8 model. Logs and software counters do not prove per-step physical DMA overlap. Owned processes and leases were released.

The targeted pytest -q tests/e2e/pull_request/one_card/test_hisparse_ops.py::test_direct_target_mirror_survives_legal_draft_writes cases passed on real A5 (eager and Graph). The byte oracle covers same-page disjoint layers and different live HMA pages, as well as Host durability. 26 source-level CPU contracts and focused Ruff/format/diff checks passed. The bounded local bash format.sh ci attempt exited during actionlint environment installation; full-format CI is not claimed. The measured C8 TPOT is within 0.17% of the preceding private snapshot run on this host, so no incremental speedup is attributed to removing the snapshot. The capacity boundary and final dataset accuracy remain separate.

Current-source 1M single-request check

Using the same production bytes as public f4a1dbb46 (runtime 1f3b0d42), the same A5/TP8/MTP3 C8 configuration, and service and MTP limits set to 1,048,576, one HiSparse ON request completed 1,040,384 input + 8,192 output token IDs. TPOT was 16.745 ms, steady Decode 59.196 tok/s, TTFT 682.756 s, with zero preemptions. Actual startup reported 64 GiB and 10,359 Host blocks per rank. This is a single-request ON result with reported misses; it does not establish an ON/OFF speedup or maximum concurrency. Owned model processes were reaped and the lease released.

Earlier corrected-head full-model ON/OFF N2 (source aa7f5dd)

Frozen public source aa7f5dd79, merged Core39 and unchanged native9bd, using the GLM-5.2 / TP8 / MTP3 / NZ0 / Hot10240 / private Host64GiB-per-rank configuration below. On one A5 host, OFF then ON, each with the same unscored primer and two scored 32K input / 8K output, concurrency 8 rounds; no preceding scored 16K workload, profiler or observer.

Round	OFF TPOT (ms)	ON TPOT (ms)	Increase
1	15.9679	17.6657	10.63%
2	15.9686	17.2771	8.19%
N2 mean	15.9683	17.4714	9.41%

Each arm completed 16/16 scored requests and 131072 actual output token IDs. Complete input/output digests and every MTP total/acceptance position match; no preemption was reported. Both models were reaped and leases released. Pooled sampled steady Decode TPS was 601.765 OFF / 552.371 ON (−8.21%). Mean TTFT was 33.058 / 33.044 s OFF and 36.154 / 33.701 s ON; these include scheduling and Prefill and are not isolated Prefill kernel timings.

These are reported-miss rounds: ON software misses were 11197176 / 36334232, with 25776391680 / 84542829184 estimated H2D bytes. The asynchronous 2000-step snapshot cadence does not measure exact physical DMA. The miss-path regression remains unresolved; this experiment does not qualify the zero-miss 3% target, statistical significance, dataset accuracy or maximum capacity. Private snapshot/overlap experiments are not included in this public source.

32K zero-reported-miss check (same public source aa7f5dd)

A separate same-source 32K input / 8K output, concurrency 4, OFF-then-ON N2 used the same configuration and unscored primer. OFF TPOT was 12.9206 / 12.9226 ms; ON was 13.3475 / 13.3867 ms, or +3.30% / +3.59% (N2 mean +3.45%). Each arm completed 8/8 scored requests and 65,536 actual output token IDs; complete outputs and total/per-position MTP matched. Present, settled ON miss and estimated-H2D deltas were zero in both rounds. Owned models were reaped and leases released. This public source misses the 3% zero-reported-miss target. Earlier 16K results below retain their original source/workload scope. These counters do not prove absence of physical DMA.

Earlier same-source full-model ON/OFF N2 (source c366b41)

Measured c366b41c3, merged Core 39dbf518390ddbadf05c6712ee6aefa1c9190219 including #63801, Python 3.12 / CANN 9.2.1, unchanged native SHA256 9bd146e9ebb2b1ae830d32915e46cd41855b7b04933d556d6406ef33ebe20015.

Full GLM-5.2 W4A8C8 MXFP, 78+1 layers, TP8/DP1, MTP3, NZ0, memory utilization 0.8, Hot 10240, private Host 64 GiB/rank, max length 139264, max batched tokens 8192/max sequences 32, FULL_DECODE_ONLY, prefix caching off and eager Host mirroring on.

EvalScope 1.13.0, concurrency 8, one unscored primer and two scored rounds per workload. Fixed input token IDs, HTTP bodies and arrival offsets; seed 42, temperature 0 and 8192 output tokens. Same host, ON then OFF. Formal timing had no profiler or observer.

Workload	OFF TPOT r1 / r2 (ms)	ON TPOT r1 / r2 (ms)	Increase r1 / r2	N2 mean
16K / 8K, zero reported misses	13.1759 / 13.1811	13.2841 / 13.3008	0.82% / 0.91%	0.86%
32K / 8K, reported misses	14.9552 / 14.9661	16.3767 / 16.4357	9.51% / 9.82%	9.66%

Each arm completed all 32 scored requests and 262144 actual output token IDs. Complete paired input/output digests and all MTP totals/acceptance positions match. Both owned models were reaped and leases released.

Pooled steady Decode TPS (running=8, waiting=0, flat prompt counter) is 650.95 OFF / 647.97 ON at 16K and 628.50 OFF / 573.23 ON at 32K. The zero-reported-miss 3% TPOT goal is met for these two rounds. The miss-path regression remains unresolved.

Present, settled software miss/H2D-estimate deltas are zero at 16K. At 32K, ON reports 13700712 / 39848560 misses and 31471815168 / 92883863936 estimated H2D bytes. These asynchronous software counters are not physical DMA measurements.

Request-phase Prefill histogram deltas have eight samples per round and exact computed-prompt counts. N2 means are 5.2937 s OFF / 5.4064 s ON at 16K (+2.13%) and 8.4227 s OFF / 8.8489 s ON at 32K (+5.06%). They include scheduling/overlap within the measured request phase, not isolated NPU kernel time.

Core-result reuse changes the actual auto-planned KV capacity: startup reports approximately 20.45–20.73 GiB/rank, versus 20.97 GiB/rank in the prior 0271d60db ON run. Complete outputs/MTP still match that prior ON; 32K refill counters differ. Do not attribute the change in the ratio to an isolated kernel optimization. N2 does not establish statistical significance or final dataset accuracy.

Column-first compact gather control

Commit 8d044bc1d changes three lines in two existing files: compact transfers schedule columns before request rows; gathers without count vectors keep their existing grid. Copied values, stream/lifetime dependencies and native Attention are unchanged.

A new frozen-source control used baseline a11b1b9d1 and candidate a3832cb71 on one A5 host, with the same Core39/Python 3.12/CANN 9.2.1/native9bd configuration above. This protocol had one unscored primer followed by 32K input / 8K output, concurrency 8, two rounds only; it did not score a preceding 16K workload.

TPOT (ms)	Round 1	Round 2	Mean
Baseline ON	18.3834	17.8872	18.1353
Candidate OFF	15.9589	15.9587	15.9588
Candidate ON	17.6771	17.3528	17.5150

Candidate ON was 3.84% / 2.99% lower than the initial baseline ON, or 3.42% lower by N2 means. A later repeat of unchanged a11b1b9d1 measured 17.7480 / 17.8305 ms, mean 17.7892 ms: 1.91% faster than its initial run without a code change. Against this late control, the candidate was 0.40% / 2.68% lower, or 1.54% lower by N2 means. Complete input/output digests and per-position MTP counts match. This exposes temporal or warm-state confounding; these sequential N2 runs do not establish a stable 3.42% causal model speedup. Candidate same-source ON/OFF regression remained 10.77% / 8.74%, mean 9.75%; these are reported-miss rounds, and the miss-path goal is unresolved. Every arm completed 16/16 scored requests and 131072 actual output token IDs, with complete input/output digests and all MTP totals/positions matching; no preemption was reported. Models and leases were released. The asynchronous miss counters retain their sampling-window limits and do not prove identical physical refill bytes.

The promoted compact kernel and its three launch methods match the measured candidate's ASTs. The later shared-Resident-copy/RoPE/intermediate-reuse changes retain their separate pending qualification. Focused Ruff, formatting and diff checks passed; the full-format attempt again reached its 55-second bound installing actionlint. N2 is a scoped comparison, not a statistical-significance or final-public-head model claim.

MTP Target group prefetch on A5

Same-host A/B on A5 with GLM-5.2 W4A8C8, TP8, C8 KV, FULL_DECODE_ONLY and MTP3. Both arms used the same experimental Ascend C Gather snapshot 90fb5ef9, derived from this PR at a642bf5b. The control used Core 24420b7a. The candidate applied only the Runtime scheduling change from vLLM #63801 to that Core revision.

Each arm ran one fixed EvalScope round at concurrency 8 with 32,768 prompt tokens and 8,192 output tokens per request. Both completed 8/8 requests. All 65,536 output token IDs matched, as did the MTP counters (22,488 drafts, 67,464 drafted tokens and 43,052 accepted tokens). The control and candidate reported 17,517,456 and 17,516,592 HiSparse misses, respectively.

Metric	Control	MTP group prefetch
Gather overlap with main-stream Cube per Target step	0.00 ms	1.33 ms
Event wait immediately before SFA per Target step	3.06 ms	2.20 ms
Steady Decode time per output token before profiling	17.69 ms	17.47 ms

The trace summary uses 12 Target replays on each of 8 ranks. Most of the overlap was with the SFA Cube stage. The steady Decode window improved by about 1.2% in this single round. The tested Ascend C Gather snapshot and the Runtime backport were experiment inputs. This measurement covers that configuration. The prefetch scheduling is already merged upstream in #63801 and is available when Ascend uses a compatible Core revision containing it.

Remaining work and limits

• Same-source CPU/device profiling and miss-path optimization remain in progress.
• Final-source A5 capacity, tested on the same host and source with fixed 32,768 input / 8,192 output tokens and 50 ms offsets per request: OFF C9 and ON C25 sustained full-concurrency Decode with complete outputs and no preemptions (2.78× the qualified concurrency). Their steady Decode throughputs were 534.02 and 861.82 tok/s, respectively, at different concurrency levels. OFF C10 completed but had one preemption. ON C26 completed all 26 outputs (212,992 token IDs), but had 313 preemptions and no sampled full-C26 Decode window after all first tokens, so it did not qualify. These are adjacent tested points in this workload, with one wave per capacity point. The current-source 1M single-request result above is a separate test. Owned workers were reaped and the leases released.
• The direct-mirror production source above met the 3% C4 zero-reported-miss target on both rounds. C8 still has measurable refill and submission overhead. Earlier results retain their original source/workload scope.
• Existing A5 device tests require the ready-a5 CI route. A3 is not qualified by these results.
• The automatically maintained Core footer below and measured Core39 are distinct; do not transfer these results unconditionally to another pin.
• Private resolver and AVSG experiments without stable device/model benefit are not included.

AI assistance was used for implementation, review and cleanup. Human review of the changed lines remains required.

Checklist

• Preserved Core lifecycle ownership and existing Attention arithmetic.

• Recorded workload-scoped NPU correctness and matched-model evidence.

• Reviewed changed files and removed confirmed repetition.

• Full CI passed.

• Final dataset accuracy and final-head maximum-concurrency qualification complete.

• vLLM main: vllm-project/vllm@24420b7

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request establishes a foundational Ascend backend for the vLLM HiSparse API. By decoupling this foundation from the larger optimized integration, it provides a stable, extensible base for contributors to build sparse MLA/SFA data paths. The implementation focuses on core integration, including device-side residency management and layout adaptation, while preserving existing vLLM Core interfaces for allocation and request lifecycle management.

Highlights

• Ascend HiSparse Foundation: Introduced an extensible Ascend backend foundation for HiSparse, enabling independent review and follow-up development for sparse MLA/SFA data paths.
• Device Operations: Implemented private mapped-Host registration, persistent per-group workspace management, and device-side resolver/refill/invalidation operations.
• Layout and ABI: Added a sparse MLA/SFA layout bridge using existing CANN Attention and fused RMSNorm/RoPE operations, with necessary builder/post-load ABI forwarding.
• Testing and Validation: Included source-isolated CPU boundary tests covering host offsets, ownership, and workspace isolation, alongside documentation for extension points and NPU qualification.

New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

---

Tip

💡 Consider Linking a Related Issue or RFC

Your PR title contains the [Feature] tag, indicating a bug fix or new feature.

Linking a related issue or RFC in the PR description is strongly encouraged — it gives reviewers helpful context and speeds up the review. You can use any of these keywords:

• Fixes #<issue_number>
• Closes #<issue_number>
• Resolves #<issue_number>
• Refs #<rfc_or_issue_number> (for RFCs)

🙏 Thanks for helping us keep the project well-organized!

[gemini-code-assist]

Code Review

Suggested PR Title:\n\nmarkdown\n[Attention][Feature] Add HiSparse foundation on Ascend\n\n\nSuggested PR Summary:\n\nmarkdown\n### What this PR does / why we need it?\nThis PR implements the foundational split of HiSparse on the Ascend backend, supplying an Ascend backend for the merged vLLM HiSparse API and the minimal SFA data path. It adds device operations, cache-layout adaptation, and metadata builders, as well as several unit and end-to-end tests to validate the implementation.\n\nFeedback on the review comments:\nSeveral critical and high-severity issues were identified in the code changes:\n1. In `vllm_ascend/ops/hisparse.py`, the `aclrtHostRegister` API was incorrectly defined with 4 arguments instead of 3, which could lead to undefined behavior or segfaults. It should be corrected to use the 3-argument signature and retrieve the mapped device pointer using `aclrtHostGetDevicePointer`.\n2. In `vllm_ascend/attention/sfa_hisparse.py`, a potential `RuntimeError` in `torch.searchsorted` was found when `metadata.num_input_tokens` is 0. A guard check should be added.\n3. In `vllm_ascend/ops/hisparse.py` and `vllm_ascend/ops/triton/hisparse.py`, the Triton block size for `residency_stats_kernel` needs to be capped at 1024 to prevent compilation or runtime failures on NPU when `rows > 1024`, and the kernel should be updated to support a grid-stride loop.\n\n### Does this PR introduce _any_ user-facing change?\nYes, it introduces the HiSparse feature on Ascend, which can be enabled via configuration options such as `--attention-config` and `--kv-transfer-config`.\n\n### How was this patch tested?\nThe patch includes extensive unit tests (`tests/ut/`) and end-to-end tests (`tests/e2e/`) covering CPU boundary checks, attention references, KV preprocessing, and prefill staging.\n

[ZT-AIA]

Please supplement the single-operator precision test case.

[zyz111222]

At Ascend f1007cd with CI Core 24420b7, GLM5.2 TP8 (32K/8K, C25) can repeatedly preempt a chunked-Prefill request under memory pressure. Core checks space for Hot and the next Prefill chunk together, but the old Resident KV can only be reclaimed after Hot is allocated. I recommend considering a Hot-first transition: allocate Hot, reclaim clean Resident pages, and then allocate the next chunk. A local implementation allowed all 25 requests to finish, though 3 capacity preemptions remain. I also recommend revalidating C25 capacity on the CI Core pin.

[zhenwenqi2024]

/ci-rerun

[zhenwenqi2024]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E

[zhenwenqi2024]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E

[zhenwenqi2024]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E

[weiguihua2]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E

[AuroraEmiya]

LGTM

[zhenwenqi2024]

/rerun
[Bot]: rerun completed.

Rerun (failed jobs only):

• E2E