Skip to content

[CI] Add debug logging for flaky test failures #18151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
alisonshao wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[CI] Add debug logging for flaky test failures #18151

alisonshao wants to merge 2 commits into from
+354 −47

Conversation

@alisonshao
Copy link
Collaborator

@alisonshao alisonshao commented Feb 3, 2026 •
edited
Loading

Summary

Add debug logging to capture server stdout/stderr when server fails to start, for debugging CI failures.

Tests updated:

  • python/sglang/test/vlm_utils.py: TestOpenAIMLLMServerBase (base class for VLM tests including test_vision_openai_server_a.py)
  • test/registered/openai_server/basic/test_openai_server.py: TestOpenAIServer, TestOpenAIServerv1Responses, TestOpenAIV1Rerank, TestOpenAIServerCustomLogitProcessor, TestOpenAIV1Score
  • test/registered/sampling/test_penalty.py: TestPenalty

Flaky tests being debugged:

  • test_vision_openai_server_a.py (Stage b test large 1 gpu)
  • test_openai_server.py (Stage b test small 1 gpu)
  • test_penalty.py (Stage b test small 1 gpu)

Note: test_multi_lora_backend.py uses SRTRunner (in-process) rather than popen_launch_server, so server stdout/stderr capture doesn't apply.

Changes

  • Modified test setUpClass methods to capture server output to temp files
  • Logs are printed on server startup failure for debugging

@gemini-code-assist
Copy link
Contributor

gemini-code-assist bot commented Feb 3, 2026

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@github-actions github-actions bot added the Multi-modal multi-modal language model label Feb 3, 2026
@alisonshao
Copy link
Collaborator Author

alisonshao commented Feb 3, 2026

/rerun-stage stage-b-test-large-1-gpu

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

✅ Triggered stage-b-test-large-1-gpu to run independently (skipping dependencies).

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

🔗 View workflow run

@alisonshao
Copy link
Collaborator Author

alisonshao commented Feb 3, 2026

/rerun-stage stage-b-test-large-1-gpu

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

✅ Triggered stage-b-test-large-1-gpu to run independently (skipping dependencies).

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

🔗 View workflow run

@alisonshao alisonshao force-pushed the debug-qwen2vl-context-length-test branch from a0228ee to 673511c Compare February 3, 2026 21:39
@alisonshao alisonshao changed the title [CI] Add debug logging to TestQwen2VLContextLengthServer [CI] Add debug logging for TestMiniCPMo26Server failure Feb 3, 2026
@alisonshao
Copy link
Collaborator Author

alisonshao commented Feb 3, 2026

/rerun-stage stage-b-test-large-1-gpu

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

✅ Triggered stage-b-test-large-1-gpu to run independently (skipping dependencies).

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

🔗 View workflow run

@alisonshao alisonshao force-pushed the debug-qwen2vl-context-length-test branch from 673511c to 5a9ccb7 Compare February 3, 2026 22:45
@alisonshao
Copy link
Collaborator Author

alisonshao commented Feb 3, 2026

/rerun-stage stage-b-test-large-1-gpu

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

✅ Triggered stage-b-test-large-1-gpu to run independently (skipping dependencies).

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

🔗 View workflow run

@alisonshao
Add debug logging to test_openai_server.py and test_penalty.py
58f54bb 
Capture server stdout/stderr to temp files and print them when server
fails to start, for debugging CI failures.

Tests updated:
- test_openai_server.py: TestOpenAIServer, TestOpenAIServerv1Responses,
  TestOpenAIV1Rerank, TestOpenAIServerCustomLogitProcessor, TestOpenAIV1Score
- test_penalty.py: TestPenalty
@alisonshao alisonshao changed the title [CI] Add debug logging for TestMiniCPMo26Server failure [CI] Add debug logging for flaky test failures Feb 3, 2026
@alisonshao
Copy link
Collaborator Author

alisonshao commented Feb 3, 2026

/rerun-stage stage-b-test-large-1-gpu

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

✅ Triggered stage-b-test-large-1-gpu to run independently (skipping dependencies).

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

🔗 View workflow run

@alisonshao
Copy link
Collaborator Author

alisonshao commented Feb 3, 2026

/rerun-stage stage-b-test-small-1-gpu

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

✅ Triggered stage-b-test-small-1-gpu to run independently (skipping dependencies).

@github-actions
Copy link
Contributor

github-actions bot commented Feb 3, 2026

🔗 View workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Multi-modal multi-modal language model

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@alisonshao