More consistent flag conventions for `ctest` and `run-tests.sh`

[Rangi42]

The test/run-tests.sh script takes some optional flags: --only-internal, --only-external, --only-free, --os <os>, and --installed-rgbds.

The test/CMakeLists.txt file, as invoked by ctest, uses a different convention, more like what other CMake/CTest commands do. You can specify ctest -DTESTS_RUN_NONFREE to not pass ./run-tests.sh --only-free, and specify ctest -DTESTS_OS_NAME=<os> to pass ./run-tests.sh --os <os>. However, as of #1931, you specify ctest --label-exclude external to pass ./run-tests.sh --only-internal (because of how we use LABELS "external"). (And there's no way to indirectly pass --only-external or --installed-rgbds.)

Running test/run-tests.sh --help prints usage information describing all those flags (and even with that, one could argue that a bunch of Boolean flags controlling which tests get run is not an elegant design). However, AFAIK, there's no simple way to check all of the different ways to configure our ctest.

• The ctest flags should be more consistent. Probably we'd want LABELs for every group of tests, instead of just for the "external" group.
• The ctest flags should be documented, or if there's a built-in way to print them out, we should still mention that, e.g. in CONTRIBUTING.md.
• Maybe the run-tests.sh flags should be redesigned, so instead of partially-overlapping Booleans (the --only-free tests include all of the --only-internal ones and some of the --only-external ones), there's one mechanism for specifying tests to exclude, same as ctest.
• Ideally we'd have more groups. Right now ctest reports five groups: rgbasm, rgblink, rgbfix, rgbgfx, and external. This makes it difficult to tell whether the free and/or nonfree external tests were run. I think we should at least split them into external-free and external-nonfree groups, and maybe even have one "group" per external repo (though we'd still want a concise way to exclude the three nonfree disassembly repos).

[ISSOtm]

This would be taken care of as part of #1927. Shouldn't this be merged there?

[Rangi42]

Sure! I didn't see these specifics mentioned there, but yeah, that refactor would make this moot.

[Rangi42]

I've reopened this issue because I may be able to fix it as a small change (basically undoing #1931's change which replaced -DTESTS_RUN_EXTERNAL with --label-exclude external). The desired fix for #1927 is a larger refactoring of our test.sh logic to use more ctest features, which is less like to be done quickly or at all.

[ISSOtm]

I think we can have that cake and eat it too, by having -DTESTS_RUN_EXTERNAL control whether the “external” tests are defined in CTest at all. Then we have the compile-time switch (-D...) and the runtime switch like run-tests.sh.

Note that we have a bit of tension right now, between the filtering API mirroring run-tests.sh in terms of shape (current -Ds) and mirroring it in terms of functionality (not requiring reconfiguration of the entire project to change those settings, which is what --label-exclude achieves).