Skip to content

Replace run.sh with a faster, filterable, DRYer python replacement#119

Open
thepetertessier wants to merge 4 commits intomatthewbdwyer:mainfrom
thepetertessier:make-python-system-test-runner
Open

Replace run.sh with a faster, filterable, DRYer python replacement#119
thepetertessier wants to merge 4 commits intomatthewbdwyer:mainfrom
thepetertessier:make-python-system-test-runner

Conversation

@thepetertessier
Copy link

@thepetertessier thepetertessier commented Dec 3, 2024

I made run_system_tests.py as a replacement for run.sh, with a few improvements:

  • It executes tests in parallel by default. On my system, this decreased test execution time by about 90%, going from 42.151s to 4.364s.
  • It has command-line parameters to filter tests (-k to filter by keyword and -d to filter by directory)
  • Instead of printing a dot for a success, it prints which test passed. It also gives more meaningful output on a failure, including a summary of which tests failed at the end.
  • It is much more modular. Adding the tests for the sip extension takes only a few lines of code, reusing the logic for selftests. Even with the extra features and comments, the whole file is only 350 lines (excluding the long documentation at the beginning and imports) compared to 300 lines of run.sh.

While developing this, I also found a small bug where executing tipc --pa=.../unwritable foo.tip exits with a success instead of a failure:

fpv5gr@portal06 ~/tipc/build $ touch unwritable
fpv5gr@portal06 ~/tipc/build $ chmod 444 unwritable
fpv5gr@portal06 ~/tipc/build $ ./src/tipc ../test/system/iotests/fib.tip --pa=unwritable
tipc: error: failed to open 'unwritable' for writing
fpv5gr@portal06 ~/tipc/build $ echo $?
0

So I fixed that too.

@thepetertessier thepetertessier changed the title make runtests.sh run python tests instead Replace run.sh with a more modular python replacement Dec 3, 2024
@thepetertessier thepetertessier changed the title Replace run.sh with a more modular python replacement Replace run.sh with a faster, filterable, DRYer python replacement Dec 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant