Add TRS ID support for run and workflow_job_init
#1596
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds comprehensive support for Tool Registry Service (TRS) IDs, enabling users to run workflows directly from Dockstore using GitHub repository references or full TRS URLs. Usage examples: planemo run workflow/github.com/iwc-workflows/parallel-accession-download/main job.yml planemo run #workflow/github.com/iwc-workflows/parallel-accession-download/main/v0.1.14 job.yml planemo run https://dockstore.org/api/ga4gh/trs/v2/tools/#workflow/github.com/iwc-workflows/parallel-accession-download/main/versions/v0.1.14 job.yml Key features: 1. **Multiple TRS ID formats supported**: - Short form: [#]workflow/github.com/org/repo/workflow_name[/version] - Full Dockstore URL: https://dockstore.org/api/ga4gh/trs/v2/tools/#workflow/... - Supports both with and without # prefix - Supports explicit version or auto-fetches latest from Dockstore 2. **Full TRS URL generation**: - Converts TRS IDs to full Dockstore API URLs - Format: https://dockstore.org/api/ga4gh/trs/v2/tools/#workflow/.../versions/... - Automatically queries Dockstore API for latest version when not specified - Preserves full URLs when provided directly 3. **Galaxy integration**: - Uses Galaxy's native TRS import API - Sends full trs_url to /api/workflows/upload endpoint - No local file downloads required - Workflows imported directly from Dockstore 4. **Implementation details**: - parse_trs_id(): Parses TRS IDs, fetches versions from Dockstore API - parse_trs_uri(): Handles trs:// prefixed URIs, reconstructs full URLs - import_workflow_from_trs(): Uses Galaxy's workflow import endpoint - for_runnable_identifier(): Auto-detects TRS IDs/URLs and converts to URIs - Renamed "branch" to "workflow_name" for clarity 5. **Error handling**: - Graceful fallback when version API fails - Catches all exceptions during version fetching - Uses tool ID without version as fallback 6. **Test coverage**: - 17 comprehensive unit tests covering: * TRS ID parsing with/without version * # prefix support * Full Dockstore URL support * Version auto-fetching with mocked API * Fallback behavior on API failures * Full integration with runnable_identifier All tests passing.
- Created install_shed_repos_for_workflow_id() to install tools from already-imported workflows - Updated _install_workflow() to install tools when --shed_install is enabled for TRS workflows - Fetches workflow definition from Galaxy, extracts tool requirements using ephemeris - Installs tools from toolshed using same mechanism as local workflows - Handles both install_repositories and update_repositories if install_most_recent_revision is set
mvdbeek
force-pushed
the
claude/add-trs-id-support-mefAv
branch
from
2e57e68 to
b80bb54
Compare
mvdbeek
force-pushed
the
claude/add-trs-id-support-mefAv
branch
from
b80bb54 to
52765f2
Compare
- Add TYPE_CHECKING import and GalaxyInstance type - Type import_workflow_from_trs params and return - Type _install_shed_repos_from_tools_info fully - Type install_shed_repos_for_workflow_id fully 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
jmchilton
reviewed
Jan 5, 2026
Member
|
Applied typing generated via https://github.com/jmchilton/claude-commands/blob/main/pyreview.md - sorry if any of it is incorrect. |
jmchilton
approved these changes
Jan 5, 2026
The parse_trs_id() function had incorrect logic for determining whether path components are workflow names or versions: - For 5-part IDs (workflow/github.com/org/repo/main), the 5th part is the workflow name, not a version - For 6-part IDs (workflow/github.com/org/repo/main/v1.0), the 6th part is the version - For 7-part IDs with "versions" keyword (workflow/.../main/versions/v1.0), correctly recognize "versions" as a keyword and extract the actual version This fixes all 4 failing tests in test_trs_id.py
Galaxy validates TRS URLs with a regex that requires the format:
https://.../ga4gh/trs/v2/tools/{tool_id}/versions/{version_id}
Changed parse_trs_id() to always provide a version:
- When version is explicitly specified, use it
- When fetching from Dockstore API succeeds, use the first version
- When API fails or returns no versions, use workflow_name as default
(e.g., "main") or "main" if no workflow_name
Updated test_parse_trs_id_version_fetch_failure to expect the new
behavior where we always include /versions/{default_version}.
This fixes the RequestParameterInvalidException error in Galaxy.
Changed from galaxy-workflow-dockstore-example-3 (which doesn't exist) to galaxy-workflow-dockstore-example-1/mycoolworkflow (which does exist). The new workflow: - Exists on Dockstore at the TRS endpoint - Has a "master" version that can be fetched - Should work correctly with Galaxy's TRS import
- Add @pytest.mark.skipif decorator to skip test_run_trs_id on Galaxy release_22.05, which doesn't support TRS import - Fix black formatting issue in test_trs_id.py by splitting long assertion across multiple lines
Applied improvements from code review (parts 1-3, excluding part 4): Part 1: HTTP Fixtures for Dockstore API - Add responses>=0.23.0 test dependency - Create fixture directory: tests/fixtures/dockstore/ - Record real Dockstore API responses to JSON fixtures - Refactor tests to use @responses.activate instead of mocks - Replace mock objects with real HTTP fixtures Part 2: TrsImporter Protocol (Dependency Injection) - Add TrsImporter Protocol for type safety - Create GalaxyTrsImporter implementation for Galaxy API - Create FakeTrsImporter for testing without Galaxy - Refactor import_workflow_from_trs to use Protocol - Update call sites to use GalaxyTrsImporter - Replace mock-based tests with state verification Part 3: Fix translate_alias Tests - Remove @patch decorators for translate_alias - Use real PlanemoCliContext with tmp_path - Eliminate mock objects in integration tests - Use planemo_directory instead of workspace property Benefits: - Tests use real objects with controlled inputs - Fakes record state instead of using mocks - HTTP responses are recorded from actual API - Better dependency injection pattern - More maintainable test code
mvdbeek
force-pushed
the
claude/add-trs-id-support-mefAv
branch
from
9fae584 to
9970865
Compare
jmchilton
reviewed
Jan 6, 2026
| # Mock translate_alias to return the input unchanged | ||
| mock_translate_alias.side_effect = lambda ctx, identifier, profile: identifier | ||
| ctx = planemo.cli.PlanemoCliContext() | ||
| ctx.planemo_directory = str(tmp_path) |
Member
There was a problem hiding this comment.
You do this over and over - is there not a test utility for this already? If not - I would create one and inject it via a pytest test fixture?
jmchilton
reviewed
Jan 6, 2026
| @@ -0,0 +1,356 @@ | |||
| [ | |||
Member
There was a problem hiding this comment.
I would include a script or a readme in this directory to document how the fixtures are generated.
jmchilton/galaxy@dc6a0ac#diff-03f9294fa6b1b5a82aad72854ef2b881b5de2deaac543ea957b56528d6a9e716. It isn't clear to me how I would regenerate these with updated fixtures on API changes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
So you can
importantly using a versioned galaxy TRS id will not reimport the workflow if you already have it.
Closes #1479