Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support filename coercion back to string #5004

Closed
stxue1 opened this issue Jul 2, 2024 · 1 comment · Fixed by #5028
Closed

Support filename coercion back to string #5004

stxue1 opened this issue Jul 2, 2024 · 1 comment · Fixed by #5028

Comments

@stxue1
Copy link
Contributor

stxue1 commented Jul 2, 2024

The spec expects that filenames can be represented as a string to alter things like filename extensions:
https://github.com/openwdl/wdl/blob/9c0b9cf4586508a9e6260cc5c5e562e21f625aac/SPEC.md?plain=1#L6521

We virtualize files into our own representation, so coercing back will give us a toilfile URI instead of the filename:

data_file = "toilfile:4%3A0%3Afiles%2Ffor-job%2Fkind-WDLTaskJob%2Finstance-kfz8ammc%2Ffile-c91abfb5298d4a83bad4bb00d53beb55%2Ffoo.data/e946ed9a-784f-47d3-bf74-e84a23d9056d/foo.data"

┆Issue is synchronized with this Jira Story
┆Issue Number: TOIL-1609

@adamnovak
Copy link
Member

Yeah, this is going to be tricky to implement. It looks like at least in task output sections they expect to be dealing with the same string that got coerced to a File, rather than something processed.

How will we make this work at workflow scope? We might have one decl that turns a String into a File, then several decls and tasks that depend on that File and need to use its virtualized representation, and then another decl that expects to operate on the File coerced back to the same original string, possibly in the same expression as working with the File's contents.

We could do something like moving virtualization to the boundaries of tasks, so we virtualize a version of the File that we use to pass to tasks, and then we leave a non-virtualized version around for use in decls?

Or we could tack the original non-virtualized path of the File onto it with setattr() so we can get at it later for string coercion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants