Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improves and standardizes Flakeguard summary data #1620

Merged
merged 2 commits into from
Feb 6, 2025

Conversation

kalverra
Copy link
Collaborator

@kalverra kalverra commented Feb 6, 2025


Below is a summarization created by an LLM (gpt-4-0125-preview). Be mindful of hallucinations and verify accuracy.

Why

The changes improve the handling of test runs and results parsing for the Flakeguard tool. They introduce clearer structuring and naming for test data, enhance the reliability of test result analysis by adjusting for different test outcomes (e.g., skips and panics), and refine the capturing of test output based on success criteria. Notably, these adjustments aim to provide a more accurate and insightful analysis of test behavior, particularly in edge cases.

What

  • runner.go:
    • Added MaxPassRatio to Runner struct to explicitly define the threshold for considering tests as flaky.
    • Updated RunTests to generate a summary of the test report dynamically, ensuring test outcomes like pass rates and run counts are accurately reflected in the final report.
  • runner_test.go:
    • Adjusted test cases to reflect the new logic and fields in the Runner struct, ensuring tests validate the enhanced functionality correctly.
  • Various JSON Test Data:
    • Updated test data files to match new output expectations, aiding in testing the modified logic.
  • Patch output adjustments:
    • Implemented minor fixes and improvements across various files to enhance clarity, maintainability, and accuracy in test result processing and output handling.

@kalverra kalverra requested review from sebawo and a team as code owners February 6, 2025 20:47
@kalverra kalverra marked this pull request as draft February 6, 2025 20:47
@kalverra kalverra marked this pull request as ready for review February 6, 2025 22:42
@cl-sonarqube-production
Copy link

@kalverra kalverra merged commit a37d05d into main Feb 6, 2025
56 of 58 checks passed
@kalverra kalverra deleted the flakeguardSummaryData branch February 6, 2025 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant