Skip to content

Bump vllm from 0.6.1.post2 to 0.8.4 in /examples/large_models/vllm #3411

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Apr 23, 2025

Bumps vllm from 0.6.1.post2 to 0.8.4.

Release notes

Sourced from vllm's releases.

v0.8.4

This release contains 180 commits from 84 contributors (25 new contributors!).

Highlights

This release includes important accuracy fixes for Llama4 models, if you are using it, we highly recommend you to update.

Model

  • Llama4 (#16113,#16509) bug fix and enhancements:
    • qknorm should be not shared across head (#16311)
    • Enable attention temperature tuning by default for long context (>32k) (#16439)
    • Index Error When Single Request Near Max Context (#16209)
    • Add tuned FusedMoE kernel config for Llama4 Scout, TP=8 on H100 (#16488)
    • Update to transformers==4.51.1 (#16257)
    • Added chat templates for LLaMa4 pythonic tool calling (#16463)
    • Optimized topk for topk=1(#16512)
    • Add warning for Attention backends that do not support irope yet (#16212)
  • Support Qwen3 and Qwen3MoE (#15289), smolvlm (#16017), jinaai/jina-embeddings-v3 (#16120), InternVL3 (#16495), GLM-4-0414 (#16338)

API

  • Estimate max-model-len use available KV cache memory. The error message nows hints at how to set --max-model-len (#16168)
  • Add hf_token to EngineArgs (#16093)
  • Enable regex support with xgrammar in V0 engine (#13228)
  • Support matryoshka representation / support embedding API dimensions (#16331)
  • Add bucket for request_latency, time_to_first_token and time_per_output_token (#15202)
  • Support for TorchAO quantization (#14231)

Hardware

  • Intel-Gaudi: Multi-step scheduling implementation for HPU (#12779)
  • TPU:
    • Make @​support_torch_compile work for XLA backend (#15782)
    • Use language_model interface for getting text backbone in MM (#16410)

Performance

  • DeepSeek MLA: a new merge_attn_states CUDA kernel, 3x speedup (#16173)
  • MoE: Support W8A8 channel-wise weights and per-token activations in triton fused_moe_kernel (#16366)
  • Add support to modelopt quantization of Mixtral model (#15961)
  • Enable PTPC FP8 for CompressedTensorsW8A8Fp8MoEMethod (triton fused_moe) (#16537)

V1 Engine Core

  • Enable multi-input by default (#15799)
  • Scatter and gather placeholders in the model runner (#16076)
  • Set structured output backend to auto by default (#15724)
  • Zero-copy tensor/ndarray serialization/transmission (#13790)
  • Eagle Model loading (#16035)
  • KV cache slots for eagle heads (#16370)
  • Add supports_structured_output() method to Platform (#16148)

Developer Facing

... (truncated)

Commits
  • dc1b4a6 [Core][V0] Enable regex support with xgrammar (#13228)
  • 63d2705 [Benchmark][Bugfix] Fix SonnetDataset default values in benchmark_throughput....
  • d085a44 Enable PTPC FP8 for CompressedTensorsW8A8Fp8MoEMethod (triton fused_moe) (#16...
  • f49e5af [V1][Spec Decode] KV cache slots for eagle heads (#16370)
  • 6c11ecf [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#...
  • 93e5f3c [Perf] Optimize Preparing Inputs for GPU Model Runner (#16484)
  • 70363bc Fix syntaxWarning: invalid escape sequence '\s' (#16532)
  • 3cdc576 [Misc] Delete redundant code (#16530)
  • 68bb122 [MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464)
  • d9fc8cd [V1] Enable multi-input by default (#15799)
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    You can disable automated security fix PRs for this repo from the Security Alerts page.

Bumps [vllm](https://github.com/vllm-project/vllm) from 0.6.1.post2 to 0.8.4.
- [Release notes](https://github.com/vllm-project/vllm/releases)
- [Changelog](https://github.com/vllm-project/vllm/blob/main/RELEASE.md)
- [Commits](vllm-project/vllm@v0.6.1.post2...v0.8.4)

---
updated-dependencies:
- dependency-name: vllm
  dependency-version: 0.8.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot bot added dependencies Pull requests that update a dependency file python Pull requests that update Python code labels Apr 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file python Pull requests that update Python code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants