Configure PyTorch x vLLM nightly dashboard #7614
Conversation
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
meta-cla
bot
added
the
CLA Signed
torchci/components/benchmark_v3/configs/teams/vllm/pytorch_x_vllm_config.ts
Show resolved
Hide resolved
| name: "PyTorch x vLLM Benchmark", | ||
| route: `/benchmark/v3/dashboard/${PYTORCH_X_VLLM_BENCHMARK_ID}`, | ||
| info: "PyTorch x vLLM nightly benchmark. Powered by [code](TODO) and [benchmark configs](https://github.com/pytorch/pytorch-integration-testing/tree/main/vllm-benchmarks/benchmarks)", | ||
| description: "Pytorch x vLLM nightly benchmark using vllm bench", |
There was a problem hiding this comment.
@huydhn recommend add comment that it's generated based on pinned vllm commit
There was a problem hiding this comment.
also, maybe put this under vllm section: "vLLM Benchmarks ", instead of pytorch,
maybe also update the description for the vllmx pinned pytorch description, so that we can tell the difference
There was a problem hiding this comment.
I put it under PyTorch mainly because the users of this dashboard will be PT2 compiler looking at vllm bench from PyTorch point of view. The public vLLM v1 dashboard is a separate entity with a different group of users, maybe we would deprecate this in the future if there is no interest from vLLM maintainers in keeping it.
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
A new dashboard for PyTorch x vLLM nightly benchmark after pytorch/pytorch#170466 lands
Preview
Benchmark results are showing up on https://torchci-git-pytorch-x-vllm-dashboard-fbopensource.vercel.app/benchmark/v3/dashboard/pytorch_x_vllm_benchmark?renderGroupId=main&time.start=2025-12-12T00%3A00%3A00.000Z&time.end=2025-12-19T23%3A59%3A59.999Z&lcommit.commit=7f2cac9b0c4cb1a0dc36f9c1daa7c6596c69b69a&lcommit.workflow_id=20362340534&lcommit.date=2025-12-19T23%3A00%3A00Z&lcommit.branch=setup-vllm-nightly-benchmark&rcommit.commit=7f2cac9b0c4cb1a0dc36f9c1daa7c6596c69b69a&rcommit.workflow_id=20362340534&rcommit.date=2025-12-19T23%3A00%3A00Z&rcommit.branch=setup-vllm-nightly-benchmark&lbranch=setup-vllm-nightly-benchmark&rbranch=setup-vllm-nightly-benchmark for the test run https://github.com/pytorch/pytorch/actions/runs/20362340534/job/58585533990
cc @zou3519