Skip to content

Conversation

MarkWolters
Copy link
Contributor

@MarkWolters MarkWolters commented Sep 18, 2025

This PR addresses the following issues with the run-bench GitHub workflow for regression testing:

  • Links benchmark_config input parameter to regression test arguments: This parameter was intended for use as a filter on the datasets to test with (e.g. provide a regex to run the regression tests on a subset of the default) but was not previously linked to the test execution so that all default datasets were tested regardless of the value of this argument
  • Fixes issue where output files were not created when branch name contains an illegal character. Illegal characters will now be replaced with underscores in the filename.
  • Fix for issue where report generation was not occurring in the final step because search for csv files was using an incorrect naming pattern.
  • Fix for diagnostic logging interfering with expected output of tests

Copy link
Contributor

github-actions bot commented Sep 18, 2025

If you haven't seen them already, please review CONTRIBUTIONS.md

Summary of Changes


Explain here why this PR is being submitted, it's purpose and impact for users.


Checklist before you submit for review

  • Did you include performance data for changes which may be performance impacting?
  • Did you include useful docs for any user-facing changes or features?
  • Did you include useful javadocs for developer oriented changes, explaining new concepts or key changes?
  • Did you trigger and review regression testing results against the base branch via Run Bench Main?
  • Did you adhere to the code formatting guidelines (TBD)
  • Did you group your changes for easy review, providing meaningful descriptions for each commit?
  • Did you ensure that all files contain the correct copyright header?

If you did not complete any of these, then please explain below

Copy link
Contributor

@jshook jshook left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@marianotepper marianotepper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@jshook jshook merged commit 19f60fe into main Sep 19, 2025
8 checks passed
@jshook jshook deleted the regression_enhancements branch September 19, 2025 21:06
jshook pushed a commit that referenced this pull request Sep 19, 2025
* fixed ability to filter datasets
* fix for illegal char in branch name
* fixed missing space for default arg
* fixed missing space for default arg
* reordering to prevent overwriting test results
* moving default diagnostic logging behavior to NONE and adding option to set diagnostic level
* fixed erroneous change
* fixed commented out files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants