Skip to content

fix a typo that I previously made in the dockerfile#449

Merged
evaneschneider merged 39 commits intocholla-hydro:devfrom
mabruzzo:fix-the-Dockerfile
Oct 29, 2025
Merged

fix a typo that I previously made in the dockerfile#449
evaneschneider merged 39 commits intocholla-hydro:devfrom
mabruzzo:fix-the-Dockerfile

Conversation

@mabruzzo
Copy link
Copy Markdown
Collaborator

I've confirmed locally that fixing this will let #386 start passing

mabruzzo and others added 10 commits September 14, 2025 19:09
This commit adds a recovery-strategy to tools/ci-setup-submodule.py that
gets invoked when git-lfs commonly fails (i.e. in the Jenkins workflow).
This strategy manually downloads files from GitHub's raw-urls. To avoid
overusing this functionality (and getting throttled by GitHub), we
disable the submodule initialization in all cases where it isn't
strictly necessary.
@mabruzzo
Copy link
Copy Markdown
Collaborator Author

mabruzzo commented Oct 22, 2025

I think I finally got everything working. It turns out that simply calling git submodule update --init triggered 2 distinct problems:

  1. sometimes the submodule would not be in the proper state (this problem comes up even when we disable git-lfs)
  2. sometimes git-lfs just fails

I have cases where problem 2 occurs without problem 1 and I have seen cases where both problems occur.

We adopt the following work-around:

  1. we invoke git submodule update --init while explicitly disabling git-lfs (to try to set up the submodule without risking any git-lfs errors)
  2. if the git-submodule isn’t in the appropriate state, we perform git -C <path/to/submod> restore . to fix up the submodule (we continue to explicitly disable git-lfs)
  3. Then we try to use git-lfs to try to fetch and checkout the large files (this would replace the corresponding “pointer-files” directly tracked in the repository that git-lfs maps to the large files)
  4. If the previous step failed, we manually replace the pointer files.

I just need to update some of the documentation, then we will be ready to merge

@mabruzzo
Copy link
Copy Markdown
Collaborator Author

Ok, this PR is ready for review.

To briefly summarize, the tools/ci-setup-submodule.py script handles all of the extra work to make sure things actually work properly. Its a little long, but most of the logic is essential (a lot of the code implements the backup strategy for when things fail). With that said, we could delete the _progress_bar function (it prints out a nice progress-bar when you run the script locally, but that doesn't seem to work in the CI at the moment)

@evaneschneider
Copy link
Copy Markdown
Collaborator

Looks great!

@evaneschneider evaneschneider merged commit a08bd2c into cholla-hydro:dev Oct 29, 2025
16 checks passed
@mabruzzo mabruzzo deleted the fix-the-Dockerfile branch December 17, 2025 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants