Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions .github/workflows/schedule_doc_linkcheck.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: Docs link check

on:
schedule:
# GitHub cron uses UTC. This runs every Sunday at 20:00 UTC+8.
- cron: '0 12 * * 0'
pull_request:
branches:
- 'main'
paths:
- 'docs/source/**/*.md'
workflow_dispatch:

defaults:
run:
shell: bash -el {0}

permissions:
contents: read
pull-requests: read

concurrency:
group: docs-linkcheck-${{ github.ref }}
cancel-in-progress: true

jobs:
linkcheck:
name: Sphinx linkcheck
runs-on: linux-aarch64-a2b3-0
container:
image: swr.cn-southwest-2.myhuaweicloud.com/base_image/ascend-ci/cann:8.5.1-910b-ubuntu22.04-py3.11
timeout-minutes: 30
steps:
- name: Checkout vllm-project/vllm-ascend repo
uses: actions/checkout@v6
Comment thread
MrZ20 marked this conversation as resolved.
Resolved

- name: Collect changed docs files
id: changed-docs
if: ${{ github.event_name == 'pull_request' }}
uses: tj-actions/changed-files@v47
with:
use_rest_api: true
files: |
docs/source/**/*.md

- name: Install docs dependencies
if: ${{ github.event_name != 'pull_request' || steps.changed-docs.outputs.any_changed == 'true' }}
run: |
python -m pip install --upgrade pip
pip install -r docs/requirements-docs.txt

- name: Run full Sphinx linkcheck
if: ${{ github.event_name != 'pull_request' }}
run: |
make -C docs linkcheck SPHINXOPTS="-W --keep-going"

- name: Run Sphinx linkcheck for changed docs
if: ${{ github.event_name == 'pull_request' && steps.changed-docs.outputs.any_changed == 'true' }}
env:
CHANGED_DOCS: ${{ steps.changed-docs.outputs.all_changed_files }}
run: |
set -euo pipefail

echo "Changed docs files:"
printf ' %s\n' ${CHANGED_DOCS}

sphinx-build -b linkcheck -W --keep-going \
docs/source docs/_build/linkcheck ${CHANGED_DOCS}

- name: Upload linkcheck reports
if: ${{ always() && github.event_name != 'pull_request' }}
uses: actions/upload-artifact@v4
with:
name: sphinx-linkcheck-report
path: |
docs/_build/linkcheck/output.txt
docs/_build/linkcheck/output.json
if-no-files-found: warn
retention-days: 5
690 changes: 343 additions & 347 deletions docs/source/community/contributors.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions docs/source/community/user_stories/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

Read case studies on how users and developers solve real, everyday problems with vLLM Ascend

- [LLaMA-Factory](./llamafactory.md) is an easy-to-use and efficient platform for training and fine-tuning large language models. It supports vLLM Ascend to speed up inference since [LLaMA-Factory#7739](https://github.com/hiyouga/LLaMA-Factory/pull/7739), gaining 2x performance enhancement in inference.
- [LLaMA-Factory](./llamafactory.md) is an easy-to-use and efficient platform for training and fine-tuning large language models. It supports vLLM Ascend to speed up inference since [LLaMA-Factory#7739](https://github.com/hiyouga/LlamaFactory/pull/7739), gaining 2x performance enhancement in inference.

- [Huggingface/trl](https://github.com/huggingface/trl) is a cutting-edge library designed for post-training foundation models using advanced techniques like SFT, PPO and DPO. It uses vLLM Ascend since [v0.17.0](https://github.com/huggingface/trl/releases/tag/v0.17.0) to support RLHF on Ascend NPUs.

- [MindIE Turbo](https://pypi.org/project/mindie-turbo) is an LLM inference engine acceleration plugin library developed by Huawei on Ascend hardware, which includes self-developed LLM optimization algorithms and optimizations related to the inference engine framework. It supports vLLM Ascend since [2.0rc1](https://www.hiascend.com/document/detail/zh/mindie/20RC1/AcceleratePlugin/turbodev/mindie-turbo-0001.html).

- [GPUStack](https://github.com/gpustack/gpustack) is an open-source GPU cluster manager for running AI models. It supports vLLM Ascend since [v0.6.2](https://github.com/gpustack/gpustack/releases/tag/v0.6.2). See more GPUStack performance evaluation information at [this link](https://mp.weixin.qq.com/s/pkytJVjcH9_OnffnsFGaew).

- [verl](https://github.com/volcengine/verl) is a flexible, efficient, and production-ready RL training library for LLMs. It uses vLLM Ascend since [v0.4.0](https://github.com/volcengine/verl/releases/tag/v0.4.0). See more information on [verl x Ascend Quickstart](https://verl.readthedocs.io/en/latest/ascend_tutorial/quick_start/ascend_quick_start.html).
- [verl](https://github.com/verl-project/verl) is a flexible, efficient, and production-ready RL training library for LLMs. It uses vLLM Ascend since [v0.4.0](https://github.com/verl-project/verl/releases/tag/v0.4.0). See more information on [verl x Ascend Quickstart](https://verl.readthedocs.io/en/latest/ascend_tutorial/quick_start/ascend_quick_start.html).

:::{toctree}
:caption: More details
Expand Down
4 changes: 2 additions & 2 deletions docs/source/community/user_stories/llamafactory.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

**Introduction**

[LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) is an easy-to-use and efficient platform for training and fine-tuning large language models. With LLaMA-Factory, you can fine-tune hundreds of pre-trained models locally without writing any code.
[LLaMA-Factory](https://github.com/hiyouga/LlamaFactory) is an easy-to-use and efficient platform for training and fine-tuning large language models. With LLaMA-Factory, you can fine-tune hundreds of pre-trained models locally without writing any code.

LLaMA-Factory users need to evaluate the model and perform inference after fine-tuning.

Expand All @@ -12,7 +12,7 @@ LLaMA-Factory uses Transformers to perform inference on Ascend NPUs, but the spe

**Benefits with vLLM Ascend**

With the joint efforts of LLaMA-Factory and vLLM Ascend ([LLaMA-Factory#7739](https://github.com/hiyouga/LLaMA-Factory/pull/7739)), LLaMA-Factory has achieved significant performance gains during model inference. Benchmark results show that its inference speed is now up to 2× faster compared to the Transformers implementation.
With the joint efforts of LLaMA-Factory and vLLM Ascend ([LLaMA-Factory#7739](https://github.com/hiyouga/LlamaFactory/pull/7739)), LLaMA-Factory has achieved significant performance gains during model inference. Benchmark results show that its inference speed is now up to 2× faster compared to the Transformers implementation.

**Learn more**

Expand Down
20 changes: 20 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,26 @@
# Copy llms.txt to site root so it is available as /llms.txt.
html_extra_path = ["llms.txt"]

# -- Options for linkcheck builder -------------------------------------------
Comment thread
MrZ20 marked this conversation as resolved.
Resolved

# Check external links without validating remote anchors. Many third-party
# sites render anchors dynamically, which makes anchor checks flaky in CI.
linkcheck_anchors = False
linkcheck_retries = 2
linkcheck_timeout = 15
linkcheck_workers = 10

# Example service endpoints in docs are intentionally not reachable from CI.
linkcheck_ignore = [
r"https?://localhost(:\d+)?($|/.*)",
r"https?://127\.0\.0\.1(:\d+)?($|/.*)",
r"https?://0\.0\.0\.0(:\d+)?($|/.*)",
r"https?://192\.0\.0\.1(:\d+)?($|/.*)",
Comment thread
MrZ20 marked this conversation as resolved.
Resolved
r"https?://<[^>]+>.*",
r"https://github\.com/vllm-project/vllm-ascend/issues/new/choose",
r"https://github\.com/[^/?#]+/?$",
]

READTHEDOCS_VERSION_TYPE = os.environ.get("READTHEDOCS_VERSION_TYPE")
if READTHEDOCS_VERSION_TYPE == "tag":
# remove the warning banner if the version is a tagged release
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ On Ascend, the design goal is the same as upstream static graph execution: reduc

## Prerequisites and References

- Upstream vLLM design doc for generic graph concepts: [CUDA Graphs](https://docs.vllm.ai/en/latest/design/cuda_graphs.html).
- Upstream vLLM design doc for generic graph concepts: [CUDA Graphs](https://docs.vllm.ai/en/latest/design/cuda_graphs/).
- PyTorch graph documentation for generic capture and replay semantics: [Accelerating PyTorch with CUDA Graphs](https://pytorch.org/blog/accelerating-pytorch-with-cuda-graphs/).
- Ascend user guide for operational enablement: [Graph Mode Guide](https://docs.vllm.ai/projects/ascend/en/latest/user_guide/feature_guide/graph_mode.html).
- Existing repo design note: `docs/source/developer_guide/Design_Documents/ACL_Graph.md`.
Expand Down
22 changes: 22 additions & 0 deletions docs/source/developer_guide/contribution/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -365,3 +365,25 @@ The doctest is a good way to make sure docs stay current and examples remain exe
```

This will reproduce the same environment as the CI. See [labeled_doctest.yaml](https://github.com/vllm-project/vllm-ascend/blob/main/.github/workflows/labeled_doctest.yaml).

### Run docs link check

You can validate external links in the Sphinx docs locally with:

```bash
make -C docs linkcheck SPHINXOPTS="-W --keep-going"
```

To check links in a specific Markdown file, pass the file to `sphinx-build`.
For example, to check only `docs/source/user_guide/release_notes.md`:

```bash
cd docs
sphinx-build -b linkcheck -W --keep-going \
source _build/linkcheck source/user_guide/release_notes.md
```

The detailed report will be written to:

- `docs/_build/linkcheck/output.txt`
- `docs/_build/linkcheck/output.json`
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ This section introduces how to perform performance testing using the benchmark s

### 3.1 Dataset

VLLM supports a variety of [datasets](https://github.com/vllm-project/vllm/blob/main/vllm/benchmarks/datasets.py).
VLLM supports a variety of [datasets](https://github.com/vllm-project/vllm/blob/main/vllm/benchmarks/datasets/datasets.py).

<style>
th {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,9 +128,9 @@ msgstr ""
#: ../../source/tutorials/models/GLM4.x.md:27
msgid ""
"`Method of Quantify`: [quantization "
"scheme](https://blog.csdn.net/qq_37368095/article/details/156429653?spm=1011.2124.3001.6209)."
"scheme](https://ai.gitcode.com/Ascend-SACT/GLM-4.5-w8a8)."
" You can use these methods to quantify the model."
msgstr "`量化方法`:[量化方案](https://blog.csdn.net/qq_37368095/article/details/156429653?spm=1011.2124.3001.6209)。您可以使用这些方法对模型进行量化。"
msgstr "`量化方法`:[量化方案](https://ai.gitcode.com/Ascend-SACT/GLM-4.5-w8a8)。您可以使用这些方法对模型进行量化。"

#: ../../source/tutorials/models/GLM4.x.md:29
msgid ""
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tutorials/models/GLM4.x.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Refer to [feature guide](../../user_guide/feature_guide/index.md) to get the fea
- `GLM-4.5-w8a8-with-float-mtp`(Quantized version with mtp): [Download model weight](https://modelers.cn/models/Modelers_Park/GLM-4.5-w8a8).
- `GLM-4.6-w8a8`(Quantized version without mtp): [Download model weight](https://modelers.cn/models/Modelers_Park/GLM-4.6-w8a8). Because vllm does not support GLM4.6 mtp in October, we do not provide an mtp version. Last month, it was supported; you can use the following quantization scheme to add mtp weights to the quantized weights.
- `GLM-4.7-w8a8-with-float-mtp`(Quantized version without mtp): [Download model weight](https://modelscope.cn/models/Eco-Tech/GLM-4.7-W8A8-floatmtp).
- `Method of Quantify`: [quantization scheme](https://blog.csdn.net/qq_37368095/article/details/156429653?spm=1011.2124.3001.6209). You can use these methods to quantify the model.
- `Method of Quantify`: [quantization scheme](https://ai.gitcode.com/Ascend-SACT/GLM-4.5-w8a8). You can use these methods to quantify the model.

It is recommended to download the model weight to the shared directory of multiple nodes, such as `/root/.cache/`.

Expand Down
2 changes: 1 addition & 1 deletion docs/source/tutorials/models/GLM5.md
Original file line number Diff line number Diff line change
Expand Up @@ -1367,7 +1367,7 @@ Refer to [Using AISBench for performance evaluation](../../developer_guide/evalu

### Using vLLM Benchmark

Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/contributing/benchmarks.html) for more details.
Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/contributing/) for more details.

## Best Practices

Expand Down
2 changes: 1 addition & 1 deletion docs/source/tutorials/models/Kimi-K2.5.md
Original file line number Diff line number Diff line change
Expand Up @@ -773,7 +773,7 @@ Refer to [Using AISBench for performance evaluation](../../developer_guide/evalu

Run performance evaluation of `Kimi-K2.5-w4a8` as an example.

Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/contributing/benchmarks.html) for more details.
Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/benchmarking/) for more details.

There are three `vllm bench` subcommands:

Expand Down
2 changes: 1 addition & 1 deletion docs/source/tutorials/models/Qwen3.5-27B.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ Refer to [Using AISBench for performance evaluation](../../developer_guide/evalu

Run performance evaluation of `Qwen3.5-27B-w8a8` as an example.

Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/contributing/benchmarks.html) for more details.
Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/benchmarking/) for more details.

There are three `vllm bench` subcommands:

Expand Down
2 changes: 1 addition & 1 deletion docs/source/tutorials/models/Qwen3.5-397B-A17B.md
Original file line number Diff line number Diff line change
Expand Up @@ -586,7 +586,7 @@ Refer to [Using AISBench for performance evaluation](../../developer_guide/evalu

Run performance evaluation of `Qwen3.5-397B-A17B-w8a8` as an example.

Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/contributing/benchmarks.html) for more details.
Refer to [vllm benchmark](https://docs.vllm.ai/en/latest/benchmarking/) for more details.

There are three `vllm bench` subcommands:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ A **disaggregated encoder** runs the vision-encoder stage of a multimodal LLM in
* In-process encoders confine reuse to a single worker.
* A remote, shared cache lets any worker retrieve existing embeddings, eliminating redundant computation.

Design doc: <https://docs.google.com/document/d/1aed8KtC6XkXtdoV87pWT0a8OJlZ-CpnuLLzmR8l9BAE>
Design doc: <https://docs.google.com/document/d/1aed8KtC6XkXtdoV87pWT0a8OJlZ-CpnuLLzmR8l9BAE/edit>

---

Expand Down
2 changes: 1 addition & 1 deletion docs/source/user_guide/feature_guide/kv_pool.md
Original file line number Diff line number Diff line change
Expand Up @@ -1147,7 +1147,7 @@ etcdctl --endpoints "${ETCD_IP}:2379" get key
```

For production environments, refer to the official etcd clustering
documentation: <https://etcd.io/docs/current/op-guide/clustering/>
documentation: <https://etcd.io/docs/v3.7/op-guide/clustering/>

### Start Datasystem Worker

Expand Down
6 changes: 3 additions & 3 deletions docs/source/user_guide/feature_guide/lora.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

## Overview

Like vLLM, vllm-ascend supports LoRA as well. The usage and more details can be found in [vLLM official document](https://docs.vllm.ai/en/latest/features/lora.html).
Like vLLM, vllm-ascend supports LoRA as well. The usage and more details can be found in [vLLM official document](https://docs.vllm.ai/en/latest/features/lora/).

You can refer to [Supported Models](https://docs.vllm.ai/en/latest/models/supported_models.html#list-of-text-only-language-models) to find which models support LoRA in vLLM.
You can refer to [Supported Models](https://docs.vllm.ai/en/latest/models/supported_models/) to find which models support LoRA in vLLM.

You can run LoRA with ACLGraph mode now. Please refer to [Graph Mode Guide](./graph_mode.md) for a better LoRA performance.

Expand All @@ -25,4 +25,4 @@ vllm serve meta-llama/Llama-2-7b \

## Custom LoRA Operators

We have implemented LoRA-related AscendC operators, such as bgmv_shrink, bgmv_expand, sgmv_shrink and sgmv_expand. You can find them under the "csrc/kernels" directory of [vllm-ascend repo](https://github.com/vllm-project/vllm-ascend.git).
We have implemented LoRA-related AscendC operators, such as bgmv_shrink, bgmv_expand, sgmv_shrink and sgmv_expand. You can find them under the "csrc/kernels" directory of [vllm-ascend repo](https://github.com/vllm-project/vllm-ascend).
2 changes: 1 addition & 1 deletion docs/source/user_guide/feature_guide/quantization.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,4 +158,4 @@ python -m vllm.entrypoints.api_server \

- [ModelSlim Documentation](https://gitcode.com/Ascend/msit/blob/master/msmodelslim/README.md)
- [LLM-Compressor GitHub](https://github.com/vllm-project/llm-compressor)
- [vLLM Quantization Guide](https://docs.vllm.ai/en/latest/quantization/)
- [vLLM Quantization Guide](https://docs.vllm.ai/en/latest/features/quantization/)
Loading
Loading