Skip to content

Commit 2a591fd

Browse files
authored
Fix the filter for broken tests in maximum test detection (#10547)
The original implementation didn't correctly filter broken tests. Ouput before: ``` Framework cutelyst defines 14 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework ffead-cpp defines 14 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework go defines 11 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework falcon defines 11 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework officefloor defines 11 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework act defines 11 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework micronaut defines 12 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework pippo defines 12 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework php defines 11 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework ubiquity defines 12 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. ``` Output after: ``` Framework cutelyst defines 14 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework falcon defines 11 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework micronaut defines 12 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework pippo defines 12 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. Framework php defines 11 tests in benchmark_config.json (max is 10). Contact maintainers and remove deprecated or discarded ones to make room. ```
1 parent 5503e47 commit 2a591fd

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

toolset/utils/metadata.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -184,7 +184,7 @@ def parse_config(self, config, directory):
184184
# Loop over them and parse each into a FrameworkTest
185185
for test in config['tests']:
186186

187-
tests_to_run = [name for (name, keys) in test.items()]
187+
tests_to_run = test.keys()
188188

189189
if "default" not in tests_to_run:
190190
log("Framework %s does not define a default test in benchmark_config.json"
@@ -193,8 +193,8 @@ def parse_config(self, config, directory):
193193

194194
# Check that each framework does not have more than the maximum number of tests
195195
maximum_tests = 10
196-
non_broken_tests_filter = lambda test: (not hasattr(test, "tags")) or ("broken" not in test.tags)
197-
non_broken_tests_to_run = list(filter(non_broken_tests_filter, tests_to_run))
196+
non_broken_tests_filter = lambda test: (not ("tags" in test.keys() and ("broken" in test["tags"])))
197+
non_broken_tests_to_run = list(filter(non_broken_tests_filter, test.values()))
198198
if len(non_broken_tests_to_run) > maximum_tests:
199199
message = [
200200
"Framework %s defines %s tests in benchmark_config.json (max is %s)."

0 commit comments

Comments
 (0)