-
Notifications
You must be signed in to change notification settings - Fork 6.3k
Speed up CI builds with ccache (seeded from develop) #16385
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
|
Thank you for your contribution to the Solidity compiler! A team member will follow up shortly. If you haven't read our contributing guidelines and our review checklist before, please do it now, this makes the reviewing process and accepting your contribution smoother. If you have any questions or need our help, feel free to post them in the PR or talk to us directly on the #solidity-dev channel on Matrix. |
5897d83 to
1d6e22d
Compare
scripts/ci/build_win.sh
Outdated
| export CCACHE_NOHASHDIR=1 | ||
| export CCACHE_COMPILERTYPE=msvc | ||
| # Hard-coded MSVC cl.exe path | ||
| CCACHE_COMPILER="C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
won't that very easily break?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean the hard-coded path?
I cannot find a reliable way to find that path.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
vswhere.exe might be able to do that for you (or vcvarsall or something)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clonker
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
some preliminary comments. very nice work altogether! i was thinking whether we should perhaps have another environment variable that globally enables or disables ccache entirely. that way, if there are problems, we can quickly switch it off.
.circleci/config.yml
Outdated
|
|
||
| # On `ci-ccache` and `breaking` we intentionally do NOT restore any ccache, so every run starts from scratch. | ||
| # Tags always skip ccache. Other branches restore the most recent cache produced on `ci-ccache`. | ||
| - when: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if this condition can be added to a command, maybe in ensure_ccache_key, and we could pass the list of branches to skip as parameters. In this way we can avoid some repetition in the config.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good idea! I did the same also for saving save_ccache_if_develop
| set -ex | ||
|
|
||
| ROOTDIR="$(dirname "$0")/../.." | ||
| ROOTDIR="$(realpath "$(dirname "$0")/../..")" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this change necessary?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because of CCACHE_BASEDIR,it needs to be an absolute path to keep ccache hash paths consistent across runs. it's probably not needed but it's cleaner imo
| mv cmake-3.27.4-windows-x86_64 "$PSScriptRoot\..\deps\cmake" | ||
| Remove-Item cmake.zip | ||
|
|
||
| Invoke-WebRequest -URI "https://github.com/ccache/ccache/releases/download/v4.12.2/ccache-4.12.2-windows-x86_64.zip" -OutFile ccache.zip |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not directly related to this PR, but would it be better to define the versions as variables? That would make them easier to update in the future and help avoid potential mistakes when changing paths after a version bump.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to tackle this in this PR?
For the purpose of testing instead of creating the cache from
develop, the cache is created by this branchci-ccache. We can later create another PR starting from this branch to see if the builds are indeed using the cache generated byci-ccache, before merging this PR let's remember to fix the occurrences ofci-ccachein the code.Results
In this PR, I edited a single file and the CI is successfully using ccache for all build job. These are the results of the speedup:
In total we save ~23 minutes (or around 2$) each run.