Skip to content

Add e2e test for multiport InferencePool enhancement #1885

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
+367 −42
Open

Add e2e test for multiport InferencePool enhancement #1885

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
ab435f8
Routine checkpoint.
RyanRosario Nov 18, 2025
6b1055c
Routine checkpoint.
RyanRosario Nov 18, 2025
f1e2e55
Merge branch 'mpece-2' of github.com:RyanRosario/gateway-api-inferenc…
RyanRosario Nov 19, 2025
9ba982f
Add verifyTraffic for multiport InfPool e2e test
RyanRosario Nov 19, 2025
772d499
Add verifyTraffic for multiport InfPool e2e test
RyanRosario Nov 20, 2025
dd7887f
Switch tests back to using llmd vllm simulator
RyanRosario Nov 22, 2025
cf949ed
make sure all metrics from each rank are present
RyanRosario Nov 24, 2025
24d2c5d
add verifyMetics logic
RyanRosario Nov 24, 2025
25cd366
add verifyMetics logic
RyanRosario Nov 24, 2025
49a3158
Merge branch 'mpece-2' of github.com:RyanRosario/gateway-api-inferenc…
RyanRosario Nov 25, 2025
411c258
Merge branch 'main' into mpece-2
RyanRosario Nov 25, 2025
d9f8c13
remove stray debug trace
RyanRosario Nov 25, 2025
c2bd9f3
address linter error
RyanRosario Nov 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions hack/test-e2e.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,6 @@ else
fi
fi

kubectl delete crd inferencepools.inference.networking.k8s.io --ignore-not-found
Copy link
Contributor

@nirrozenbaum nirrozenbaum Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this script deletes the CRD?

Copy link
Author

@RyanRosario RyanRosario Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be a timing issue in my local environment. Aborting or failing tests (moreso the former) followed by immediate re-execution sees an InferencePool that hasn't been torn down yet. I would imagine other devs experience this as well so I added it, but I can remove it if not a good practice.

echo "Found an active cluster. Running Go e2e tests in ./epp..."
go test ./test/e2e/epp/ -v -ginkgo.v
6 changes: 4 additions & 2 deletions test/e2e/epp/e2e_suite_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ import (

const (
// defaultCurlTimeout is the default timeout for the curl command to get a response.
defaultCurlTimeout = 30 * time.Second
defaultCurlTimeout = 120 * time.Second
Copy link
Contributor

@nirrozenbaum nirrozenbaum Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 min for curl timeout is a bit long.
why do we need to increase the 30 sec timeout?

Copy link
Author

@RyanRosario RyanRosario Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right, though 30 was too short. When generating traffic in verifyMetrics I was getting timeouts (curl 28). I implemented retries but was still getting timeouts with 30s -- only increasing the timeout worked. I am not confident that increasing the number of retries alone would resolve it. I am open to suggestions.

// defaultCurlInterval is the default interval to run the test curl command.
defaultCurlInterval = time.Second * 5
// defaultNsName is the default name of the Namespace used for tests. Can override using the E2E_NS environment variable.
Expand Down Expand Up @@ -317,7 +317,9 @@ func createEnvoy(testConfig *testutils.TestConfig, filePath string) {

// createInferExt creates the inference extension resources used for testing from the given filePath.
func createInferExt(testConfig *testutils.TestConfig, filePath string) {
inManifests := testutils.ReadYaml(filePath)

// This image needs to be updated to open multiple ports and respond.
inManifests := testutils.ReadYaml(filePath) // Modify inference-pool.yaml
ginkgo.By("Replacing placeholders with environment variables")
outManifests := []string{}
replacer := strings.NewReplacer(
Expand Down
Loading