Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

succeeded.log should come from a temporary directory and not the working directory #5086

Open
adamnovak opened this issue Sep 11, 2024 · 0 comments
Labels

Comments

@adamnovak
Copy link
Member

adamnovak commented Sep 11, 2024

The code to make succeeded.log in the job store on completion makes that file in the working directory of the leader, and then deletes it.

toil/src/toil/leader.py

Lines 299 to 305 in 82c787c

logName = 'failed.log' if fail else 'succeeded.log'
localLog = os.path.join(os.getcwd(), logName)
open(localLog, 'w').close()
self.jobStore.import_file('file://' + localLog, logName, hardlink=True)
if os.path.exists(localLog): # Bandaid for Jenkins tests failing stochastically and unexplainably.
os.remove(localLog)

This is going to race on the delete if multiple Toil workflows are running at once (hence the note about Jenkins test failures), and is generally astonishing behavior. What if the user had important data in a file named succeeded.log?

If we need a file with that name outside the job store to put into it, we should make a temporary directory to hold it.

┆Issue is synchronized with this Jira Story
┆Issue Number: TOIL-1642

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant