DX: Faster feedback loop when running regression tests

[staabm]

when working in PHPUnit itself it is helpful to run previously errors first to shorten the feedback loop.

I also tried random order, but it seems to trigger a warning when running tests, therefore I just went with defects for now.

[sebastianbergmann]

I am not comfortable with this change because I am not happy with how test reordering currently works. Let me explain.

Currently, if we do not configure executionOrder in the XML configuration, all tests from the unit suite are run first, followed by all tests from the end-to-end suite. This means that all the fast tests are run first, followed by all the slow tests. I would like to keep this and only reorder the tests so that defects are run first. This is what you want to achieve with this pull request.

However, with executionOrder set to defects, all incomplete and skipped tests are run first, as expected. Then the slow tests are executed before the fast tests (which is unexpected). In my opinion, this is wrong and definitely not what I want (see above).

On a side note, I am no longer convinced that incomplete or skipped tests should be reordered when the defects reordering strategy is used. In this context, incomplete or skipped tests are not defective.

Additionally, neither the XML configuration file nor the --order-by CLI option allows for all sensible combinations of test reordering strategies. For example, "resolve dependencies" should be combinable with "defects first" and "order by duration".

In short, this simple pull request has made it clear to me that the current state of test reordering is poor.