Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation for using action-versions on self-hosted runners #10

Closed
alec-drw opened this issue Oct 31, 2023 · 7 comments
Closed

Documentation for using action-versions on self-hosted runners #10

alec-drw opened this issue Oct 31, 2023 · 7 comments

Comments

@alec-drw
Copy link

Hello! This project helps address many issues I have seen regarding rate limiting on self-hosted runners when checking out certain actions on GHES, even when using the "local" actions organization that has the default actions. However, there is no documentation on how to make use of this project - can the README or some documentation provide a high level "how to" set this up on runners built and maintained externally from the hosted runners?

@TingluoHuang
Copy link
Member

Depends on where the rate limiting coming from, this cache might not work for you.
Based on my understanding, there are 2 potential API calls.

  • Ask service to resolve an action to its SHA and verify policy around using the action (most likely get rate limited)
  • Download the tar.gz/zip for the action (improved by this repo)

I think it might be better to figure out which case you are seeing before adopting the cache in this repository.

@alec-drw
Copy link
Author

alec-drw commented Nov 8, 2023

@TingluoHuang it seems some jobs get secondary rate limited, for example we have a workflow that spawns ~500 jobs and makes at least 468 API calls to our GHES and we have a secondary rate limit at 500 rps

Error: API rate limit exceeded while resolving action `actions/checkout@v4`.. API rate limit exceeded while resolving action `actions/download-artifact@v3`.. API rate limit exceeded while resolving action `actions/upload-artifact@v3`.. API rate limit exceeded while resolving action `aws-actions/configure-aws-credentials@v4`.
##[debug]GitHub.DistributedTask.WebApi.UnresolvableActionDownloadInfoException: API rate limit exceeded while resolving action `actions/checkout@v4`.. API rate limit exceeded while resolving action `actions/download-artifact@v3`.. API rate limit exceeded while resolving action `actions/upload-artifact@v3`.. API rate limit exceeded while resolving action `aws-actions/configure-aws-credentials@v4`.
##[debug]   at GitHub.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
##[debug]   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
##[debug]   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
##[debug]   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
##[debug]   at GitHub.Runner.Worker.ActionManager.GetDownloadInfoAsync(IExecutionContext executionContext, List`1 actions)
##[debug]   at GitHub.Runner.Worker.ActionManager.GetDownloadInfoAsync(IExecutionContext executionContext, List`1 actions)
##[debug]   at GitHub.Runner.Worker.ActionManager.PrepareActionsRecursiveAsync(IExecutionContext executionContext, PrepareActionsState state, IEnumerable`1 actions, Int32 depth, Guid parentStepId)
##[debug]   at GitHub.Runner.Worker.ActionManager.PrepareActionsAsync(IExecutionContext executionContext, IEnumerable`1 steps, Guid rootStepId)
##[debug]   at GitHub.Runner.Worker.JobExtension.InitializeJob(IExecutionContext jobContext, AgentJobRequestMessage message)

So putting the official actions on the runner may get us around this

@alec-drw
Copy link
Author

alec-drw commented Nov 9, 2023

Found the commit that adds support for this in runners (actions/runner@f57ecd8). My approach is to create a PV, mount it at /opt/hostedactioncache and populate that directory with the actions using the scripts here, and set ACTIONS_RUNNER_ACTION_ARCHIVE_CACHE to that location.

Going to give this a try Monday, if this sounds way off please let me know! Happy to write something for the ARC repo or similar based on the outcome.

@TingluoHuang
Copy link
Member

@alec-drw you are getting API rate limit exceeded while resolving action, the cache will help you skip the actual tarball download, but it won't skip the resolving action part, which means you will still have chance to hit rate limiting. 😢

@alec-drw
Copy link
Author

@TingluoHuang Thanks for clarifying - regarding the ticket, is providing usage documentation something you still may do? I've noticed that even with my changes noted above the logs do not point to it checking the cache before downloading the action

@mildmojo
Copy link

mildmojo commented Mar 9, 2024

I think I have the same question; I can't tell what this repository is, or how it's used for a self-hosted runner installation.

  • Is this a breakout repo for a feature that's baked into the runner software or official docker image?
  • Is this an addon you'd add to a custom runner deployment?
  • If it's an addon, how do you use it? Do you call one of the scripts before runner startup (which one(s)?), or do you clone the repo to a special location where the runner automatically checks and executes it on startup?

The promise of caching common actions sounds great, though, if it's not something built into the runner already!

@TingluoHuang
Copy link
Member

https://github.com/actions/action-versions/blob/main/README.md#how-to-use-this-in-the-self-hosted-runner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants