Parallel running of test folders with sequential running inside

[dfrydrych]

I use Robot Framework to test a larger application that consists of several independent modules. Tests (*.robot files) for individual modules are stored in separate directories, see the schema below. In a standard (sequential) run, the files are run sequentially, sorted by their names. The first test is initialization, which prepares data in the database that the application uses. Other tests then work with this data. I also use test dependencies to eliminate cases where initialization fails with an error.

There are already quite a lot of tests and all tests take several hours.
It would be possible to speed them up by running tests for individual modules in parallel, while maintaining the order of the tests in the directory corresponding to one module.

The solution would be to run several RobotFrameworks (in parallel), each on only one directory corresponding to one module. But the results will then also be in separate files. However, it would be ideal to have only one file containing tests for all modules.

Therefore, I am trying to configure pabot, but this option is not mentioned in the documentation and I have not found a reasonable answer anywhere (using Suite setup, Tear down does not solve this problem).
It seems to me that either I am using Robot Framework incorrectly or this trivial task is not described in the documentation.

Thank you for your help.

---

Schema of folders and tests

tests/

moduleA/

initialization.robot
testA.robot
testB.robot

moduleB/

initialization.robot
test1.robot
test2.robot

moduleC/

initialization.robot
testA.robot
testB.robot

[joonaskuisma]

Hi!

If I understood correctly, I think the --ordering file option in Pabot is what you’re looking for. It allows you to group tests, and in particular its #DEPENDS feature is likely useful. You need to define the test suites (initialization) that you want to run first as prerequisites for the other tests using #DEPENDS.

It’s worth checking the pabot's documentation for more details, and feel free to ask if you need further help.

[Brownies]

@dfrydrych I think if you rename your initialization.robot files to __init__.robot pabot will do what you want. See Suite initialization files in Robot's user guide for more info.

If you need more granular control of test order the ordering file is the way to go.

[joonaskuisma]

I am cleaning Pabot's issues and this looks more like a usage / Q&A topic, so I’m moving it to Discussions → Q&A where it’s easier for others to find and benefit from the answer.

Feel free to continue the conversation there 👍