Implement Declarative Testing for Workflow Behaviors #18542
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
test_workflows.py
has long grown unwieldy and many tests are just running workflows and checking outputs. I think these should be broken out and done in a declarative manner. I was also pondering #18541 and trying to think about how and where to describe the expected behavior and I didn't think just adding another test to the monster file was the right approach. I think there are limitations to what is being done with framework test tools (added in 569c5db) but they are better than a bunch of Python test cases I believe. In this PR, I have created a framework for running workflow framework tests to match framework tool tests. I've migrated a few workflow tests to the framework and I have implemented my desired test for #18541.The syntax is not exactly what Planemo uses for workflow tests... but it is close and I think we can converge there over time. The biggest change is just input descriptions - the workflow tests grew up around automatically generated fixtures and Planemo is optimized for users specifying actual files. So the input specification is different - but they already different in the string-ified YAML throughout the test framework. I've implemented the
elements
syntax introduced by @mvdbeek for Planemo in galaxyproject/planemo#1417.How to test the changes?
(Select all options that apply)
License