Document writing a new conformance test

[mr-c]

Partial fix for common-workflow-language/cwltest#59 and common-workflow-language/cwltest#61 (some of this text should be copied over to https://github.com/common-workflow-language/cwltest/ later)

Partial fix for common-workflow-language/common-workflow-language#629

Fixes common-workflow-language/common-workflow-language#740

[rupertnash]

OK - I'm going to try to write the conformance test needed to close #86 following these instructions like the literal minded pedant that I am.

First thing I wonder is, do I need to install cwltest or does the run_test.sh script do that for me? Do I need to do any setup? If so, a requirements.txt would be nice and a reminder to the less experienced that they should create a venv for this.

[mr-c]

First thing I wonder is, do I need to install cwltest

Yes, thanks for the reminder. I added basic instructions for installing using a virtualenv. I'll mention the conda package once bioconda/bioconda-recipes#30871 is merged and released

[rupertnash]

We had a brief discussion about terminology: we converged on a summary that:

A CWL document contains one or more CWL descriptions

[rupertnash]

What do we do about path for files and directories? I can see two cases where it's specified as path: Any and no cases where it's set to anything else.

[rupertnash]

What are the rules for adding an id to a conf. test? In the main conformance_tests.yaml, they are integers but in the imported ones they are strings $prefix-$local_integer

Is an id even required?

[mr-c]

What are the rules for adding an id to a conf. test? In the main conformance_tests.yaml, they are integers but in the imported ones they are strings $prefix-$local_integer

We are migrating off of id to only use the labels: common-workflow-language/cwltest#110

Is an id even required?

No, id is optional: https://github.com/common-workflow-language/cwltest/blob/097088ecfd599c7a096e9d3fc05d458456d2a641/cwltest/cwltest-schema.yml#L8

[rupertnash]

This now covers the questions I had in writing a test :)

[mr-c]

Thanks @rupertnash ! I need you to review my latest clarifications again (this repo is sensitive about changes to the main branch)

[rupertnash]

The last commit further clarifies - I approve