Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dynamic DockerRequirement.dockerPull #45

Open
mr-c opened this issue Jul 18, 2024 · 1 comment
Open

dynamic DockerRequirement.dockerPull #45

mr-c opened this issue Jul 18, 2024 · 1 comment

Comments

@mr-c
Copy link
Member

mr-c commented Jul 18, 2024

We have gotten this requirement often.

https://cwl.discourse.group/t/dynamic-changing-docker-pull-registry-location-in-dockerrequirements/530
https://cwl.discourse.group/t/using-javascript-in-requirements-such-as-dockerpull/814
(and many other times)

It would also improve the WDL to CWL converter, as WDL workflows often have one or more string inputs used to specify the name of a container for some particular step(s).

However some CWL workflow runners, like Arvados, want to know the names of all containers at workflow submission time (to copy them to shared storage)

@tetron suggests the addition of a { class: ContainerName, dockerPull: user_provided_value } or similar. Then a workflow runner can scan the inputs object for class: ContainerName (and check the whole workflow DAG for any static class: ContainerNames) and handle the copying of those containers ahead of time.

@tetron
Copy link
Member

tetron commented Jul 23, 2024

Thinking a little more about this, this might also be an opportunity to redefine/migrate containers in CWL to reference the OCI image format and OCI image distribution spec for fetching/using container environments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants