Skip to content

Fix the filter for broken tests in maximum test detection #10547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
msmith-techempower merged 1 commit into from
Jan 7, 2026
+3 −3
Merged

Fix the filter for broken tests in maximum test detection #10547

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions toolset/utils/metadata.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ def parse_config(self, config, directory):
# Loop over them and parse each into a FrameworkTest
for test in config['tests']:

tests_to_run = [name for (name, keys) in test.items()]
tests_to_run = test.keys()

if "default" not in tests_to_run:
log("Framework %s does not define a default test in benchmark_config.json"
Expand All @@ -193,8 +193,8 @@ def parse_config(self, config, directory):

# Check that each framework does not have more than the maximum number of tests
maximum_tests = 10
non_broken_tests_filter = lambda test: (not hasattr(test, "tags")) or ("broken" not in test.tags)
non_broken_tests_to_run = list(filter(non_broken_tests_filter, tests_to_run))
non_broken_tests_filter = lambda test: (not ("tags" in test.keys() and ("broken" in test["tags"])))
non_broken_tests_to_run = list(filter(non_broken_tests_filter, test.values()))
if len(non_broken_tests_to_run) > maximum_tests:
message = [
"Framework %s defines %s tests in benchmark_config.json (max is %s)."
Expand Down
Loading