Skip to content

feat(scheduling): add endpoint attribute weight scorer - #2204

Open
nicole-lihui wants to merge 1 commit into
llm-d:mainfrom
nicole-lihui:feat/2201-endpoint-label-weight-scorer
Open

feat(scheduling): add endpoint attribute weight scorer#2204
nicole-lihui wants to merge 1 commit into
llm-d:mainfrom
nicole-lihui:feat/2201-endpoint-label-weight-scorer

Conversation

@nicole-lihui

@nicole-lihui nicole-lihui commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind feature

What this PR does / why we need it:

Add label-producer and endpoint-attribute-weight-scorer.

The producer exposes Pod labels as endpoint attributes, and the scorer maps discrete attribute values to normalized routing scores. This supports static preferences for heterogeneous GPUs, zones, racks, and similar endpoint groups while keeping missing or unknown values eligible.

Which issue(s) this PR fixes:

Fixes #2201

Test plan:

  • Unit tests cover configuration validation, score normalization, fallback behavior, and scorer category.
  • Hermetic integration tests verify weighted GPU routing and eligibility of missing or unknown attributes.

Release note (write NONE if no user-facing change):

Add endpoint attribute weight scoring for configurable static routing preferences.

@nicole-lihui
nicole-lihui requested a review from a team as a code owner July 27, 2026 14:03
@nicole-lihui
nicole-lihui requested review from ahg-g and elevran July 27, 2026 14:03
@github-actions github-actions Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jul 27, 2026
@nicole-lihui
nicole-lihui force-pushed the feat/2201-endpoint-label-weight-scorer branch from edc8d4b to 10e56bd Compare July 27, 2026 15:50
@nicole-lihui nicole-lihui changed the title feat(scheduling): add label weight scorer feat(scheduling): add endpoint attribute weight scorer Jul 27, 2026
Refs llm-d#2201

Signed-off-by: nicole-lihui <nicole.li@daocloud.io>
@nicole-lihui
nicole-lihui force-pushed the feat/2201-endpoint-label-weight-scorer branch from 10e56bd to a1f5325 Compare July 27, 2026 15:55
@elevran

elevran commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

thanks @nicole-lihui !
Q: given that labels are already part of the Endpoint metadata state, what is the added value of making labels into Endpoint attributes? It can probably result in more focused/granular plugins, but I wonder what other reasons might be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add a generic endpoint-label-weight-scorer for static routing preferences

2 participants