add align, chain, and net workflow

[martenson]

FOR CONTRIBUTOR:

• I have read the Adding workflows guidelines
• License permits unrestricted use (educational + commercial)
• Please also take note of the reviewer guidelines below to facilitate a smooth review process.

FOR REVIEWERS:

• .dockstore.yml: file is present and aligned with creator metadata in workflow. ORCID identifiers are strongly encouraged in creator metadata. The .dockstore.yml file is required to run tests
• Workflow is sufficiently generic to be used with lab data and does not hardcode sample names, reference data and can be run without reading an accompanying tutorial.
• In workflow: annotation field contains short description of what the workflow does. Should start with This workflow does/runs/performs … xyz … to generate/analyze/etc …
• In workflow: workflow inputs and outputs have human readable names (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless it is generally understood. Altering input or output labels requires adjusting these labels in the the workflow-tests.yml file as well
• In workflow: name field should be human readable (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless generally understood
• Workflow folder: prefer dash (-) over underscore (_), prefer all lowercase. Folder becomes repository in iwc-workflows organization and is included in TRS id
• Readme explains what workflow does, what are valid inputs and what outputs users can expect. If a tutorial or other resources exist they can be linked. If a similar workflow exists in IWC readme should explain differences with existing workflow and when one might prefer one workflow over another
• Changelog contains appropriate entries
• Large files (> 100 KB) are uploaded to zenodo and location urls are used in test file

[mvdbeek]

Is there a non-GPU flag we can use in testing ? Otherwise I'm afraid we'll have to run the GPU step elsewhere, but I worry that's going to be a pain.

[martenson]

@mvdbeek I am not sure if there is some non-gpu hack, I'll have a closer look. The test runs fine against vgp.usegalaxy.org but that is likely not very helpful here.

edit: no hack found

[mvdbeek]

I think we'll have to invent some way to say that a workflow needs to run against a live instance when it really can't run in CI. Or perhaps the better way is to setup a "only for IWC" gpu pulsar destination. Then we only launch the step that needs special stuff against the remote resource.

[github-actions]

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	1
Passed	0
Error	1
Failure	0
Skipped	0

Errored Tests

• ❌ align.ga_0

  Execution Problem:

  • Failed to run workflow, at least one job is in [error] state.
    

  Workflow invocation details

  • Invocation Messages

  • Steps

    • Step 1: TARGET_fasta_index:

      • step_state: scheduled

    • Step 2: QUERY_fasta_index:

      • step_state: scheduled

    • Step 11: chainprenet_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is new

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            inclHap	false
            pad	None
            query_reference_index_source	{"__current_case__": 1, "in_que_ref_index": {"values": [{"id": 2, "src": "hda"}]}, "query_reference_index_source_selector": "history"}
            target_reference_index_source	{"__current_case__": 1, "in_tar_ref_index": {"values": [{"id": 1, "src": "hda"}]}, "target_reference_index_source_selector": "history"}

    • Step 12: chainsort_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is new

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            sort_by	""

    • Step 13: chainne_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is new

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            inclHap	false
            minFill	None
            minScore	None
            minSpace	None
            query_reference_index_source	{"__current_case__": 1, "in_que_ref_index": {"values": [{"id": 2, "src": "hda"}]}, "query_reference_index_source_selector": "history"}
            target_reference_index_source	{"__current_case__": 1, "in_tar_ref_index": {"values": [{"id": 1, "src": "hda"}]}, "target_reference_index_source_selector": "history"}
            verbose	None

    • Step 14: netsyntenic_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is new

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"

    • Step 15: netfilter_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is new

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            minGap	None
            synteny	{"__current_case__": 1, "minSynAli": null, "minSynScore": null, "minSynSize": null, "syn_filter": "filtersyn", "syntype": "-syn"}

    • Step 16: nettoaxt_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is new

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            maxGap	None
            noSplit	false
            qChain	false

    • Step 17: axttomaf_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is new

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            q_prefix	None
            query_reference_index_source	{"__current_case__": 1, "in_que_ref_index": {"values": [{"id": 2, "src": "hda"}]}, "query_reference_index_source_selector": "history"}
            score	false
            scoreZero	false
            t_prefix	None
            target_reference_index_source	{"__current_case__": 1, "in_tar_ref_index": {"values": [{"id": 1, "src": "hda"}]}, "target_reference_index_source_selector": "history"}

    • Step 3: TARGET_Sequence:

      • step_state: scheduled

    • Step 4: QUERY_Sequence:

      • step_state: scheduled

    • Step 5: target_2bit_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is ok

          Command Line:

          • faToTwoBit '/tmp/tmpl552jumv/files/1/d/d/dataset_1ddd3eaf-895e-4cc5-b885-4d80f33168b4.dat' '/tmp/tmpl552jumv/job_working_directory/000/6/outputs/dataset_60a339ec-e8cd-47b5-9fec-b727e246f364.dat'

          Exit Code:

          • 0

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"fasta"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"

    • Step 6: kegalign_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is error

          Command Line:

          • mkdir -p "$(pwd)/work" && faToTwoBit <(gzip -cdfq '/tmp/tmpl552jumv/files/a/5/c/dataset_a5c31a61-a7c4-4514-af48-fe1da054d7d0.dat') "$(pwd)/work/ref.2bit" && faToTwoBit <(gzip -cdfq '/tmp/tmpl552jumv/files/7/9/d/dataset_79d24603-74b4-429f-aded-95f07a60488d.dat') "$(pwd)/work/query.2bit" &&   python '/tmp/shed_dir/toolshed.g2.bx.psu.edu/repos/richard-burhans/kegalign/544ea81a6914/kegalign/runner.py' --output-type tarball --output-file '/tmp/tmpl552jumv/job_working_directory/000/7/outputs/dataset_02c82e58-8f24-48e7-8e58-14ba1ed41b70.dat' --diagonal-partition --num-cpu ${GALAXY_SLOTS:-2} --tool_directory '/tmp/shed_dir/toolshed.g2.bx.psu.edu/repos/richard-burhans/kegalign/544ea81a6914/kegalign' '/tmp/tmpl552jumv/files/a/5/c/dataset_a5c31a61-a7c4-4514-af48-fe1da054d7d0.dat' '/tmp/tmpl552jumv/files/7/9/d/dataset_79d24603-74b4-429f-aded-95f07a60488d.dat'   --strand 'both'     --seed '12of19' --step '1'   --xdrop '910' --hspthresh '3000'   --ydrop '9430'   --format 'axt'   --wga_chunk_size '250000' --lastz_interval_size '10000000' --seq_block_size '400000000' --num_gpu '-1'   && python '/tmp/shed_dir/toolshed.g2.bx.psu.edu/repos/richard-burhans/kegalign/544ea81a6914/kegalign/package_output.py' --tool_directory '/tmp/shed_dir/toolshed.g2.bx.psu.edu/repos/richard-burhans/kegalign/544ea81a6914/kegalign' --format_selector 'axt'

          Exit Code:

          • 1

          Standard Error:

          • cfg.num_threads = 1
            Using 1 threads
            Error: No GPU device found!
            Error: kegalign exited with returncode 1
            

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            gapped_extension_options	{"gappedthresh": null, "notrivial": false, "ydrop": "9430"}
            output_options	{"format": {"__current_case__": 4, "axt_type": "axt", "format_selector": "axt"}}
            scoring_options	{"ambiguous_params": {"__current_case__": 0, "set_ambiguous_params_selector": "false"}, "ambiguous_selector": "x", "scoring": null}
            seeding_options	{"notransition": false, "seed": {"__current_case__": 0, "seed_selector": "12of19"}, "step": "1"}
            sequence_options	{"strand_selector": "both"}
            system_options	{"debug": false, "lastz_interval_size": "10000000", "num_gpu": "-1", "seq_block_size": "400000000", "wga_chunk_size": "250000"}
            ungapped_extension_options	{"hspthresh": "3000", "noentropy": false, "xdrop": "910"}

    • Step 7: query_2bit_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is ok

          Command Line:

          • faToTwoBit '/tmp/tmpl552jumv/files/4/1/5/dataset_415d5664-ac16-4346-a410-651ca2d7d4b0.dat' '/tmp/tmpl552jumv/job_working_directory/000/9/outputs/dataset_78d6bb60-42d1-4027-b000-6668fd70669d.dat'

          Exit Code:

          • 0

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"fasta"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"

    • Step 8: batched_lastz_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is paused

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"

    • Step 9: axtchain_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is new

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            linear_gap_options	{"__current_case__": 0, "linear_gap": "medium"}
            minScore	None
            scoreScheme	None

    • Step 10: chainantirepeat_step:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is new

          Traceback:

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"36be61022cde11f0a4526045bd08c38f"
            chromInfo	"/tmp/tmpl552jumv/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            minScore	None
            noCheckScore	None

  • Other invocation details

    • error_message

      • Failed to run workflow, at least one job is in [error] state.

    • history_id

      • 3b47e5167388cd76

    • history_state

      • error

    • invocation_id

      • 3b47e5167388cd76

    • invocation_state

      • scheduled

    • workflow_id

      • 3b47e5167388cd76