Skip to content

[Bugfix][Model] Qwen3-Omni: move cu_seqlens to GPU before VIT attention - #1063

Merged
mgehre-amd merged 1 commit into
gfx11from
matthias.qwen3-omni-cu-seqlens-gpu
Jul 27, 2026
Merged

[Bugfix][Model] Qwen3-Omni: move cu_seqlens to GPU before VIT attention#1063
mgehre-amd merged 1 commit into
gfx11from
matthias.qwen3-omni-cu-seqlens-gpu

Conversation

@mgehre-amd

@mgehre-amd mgehre-amd commented Jul 27, 2026

Copy link
Copy Markdown

Cherry-pick of upstream 540aaf2140 (vllm-project/vllm#44264, 2026-06-08) onto gfx11.

Problem

Both Qwen3-Omni-30B-A3B VLM regression configs fail engine-core init on gfx11 when the
multimodal encoder uses the TRITON_ATTN backend:

ValueError: Pointer argument cannot be accessed from Triton (cpu tensor?)
 -> RuntimeError: Engine core initialization failed

The FLASH_ATTN encoder backend does not hit this only because
flash_attn_maxseqlen_wrapper in vit_attn_wrappers.py carries a downstream-only guard
(added in cdd11a6abc, "Merge upstream vLLM code into gfx11") that copies cu_seqlens to
q.device. triton_attn_wrapper in the same file has no equivalent guard. Other VLMs
(Qwen2.5-VL / Qwen3-VL) are unaffected because they build encoder metadata through
MMEncoderAttention.maybe_recompute_cu_seqlens, which already lands the tensor on the device.

Verification

gfx1151, torch 2.11.0+rocm7.15.0a20260626, triton 3.7.1, vLLM 965d21822d.

Repro command (fails before this change, passes after):

vllm-bench.py --model cyankiwi/Qwen3-Omni-30B-A3B-Instruct-AWQ-4bit \
  --num-prompts 10 --max-model-len 4096 --input-len 512 --output-len 128 \
  --dtype float16 --trust-remote-code --target-gpu-memory-gb 28 --max-num-seqs 1 \
  --synthetic-mm --synthetic-mm-width 1024 --synthetic-mm-height 800 \
  --synthetic-mm-num-images 1 --synthetic-mm-base-items-per-request 1 \
  --mm-encoder-attn-backend TRITON_ATTN --ready-check-timeout-sec 2400 \
  -e TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=TRUE \
  -e FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE -e TORCH_BLAS_PREFER_HIPBLASLT=1

…on (vllm-project#44264)

Signed-off-by: Lanze Liu <lanzetech@gmail.com>
(cherry picked from commit 540aaf2)
@mgehre-amd
mgehre-amd requested a review from zohaibm-amd July 27, 2026 09:15
@mgehre-amd
mgehre-amd merged commit 37b2b35 into gfx11 Jul 27, 2026
7 checks passed
@mgehre-amd
mgehre-amd deleted the matthias.qwen3-omni-cu-seqlens-gpu branch July 27, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants