feat(ampup): bounded-concurrency parallel downloads#5
Open
Conversation
1bfd0cb to
c58f15a
Compare
Introduce DownloadManager with Semaphore + JoinSet for parallel artifact downloads (default 4, configurable via -j/--jobs). Includes staging directory with atomic rename for no-partial-installs, one retry per download, fail-fast cancellation, token resolution chain (explicit → gh auth token → unauthenticated), and shared rate limiter with 429/403 handling.
c58f15a to
130eef3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements bounded-concurrency parallel downloads for installing multiple components simultaneously, addressing edgeandnode/amp#1762.
DownloadManager— new module using TokioSemaphore+JoinSetfor parallel artifact downloads (default 4 concurrent, configurable via-j/--jobs)download_with_retryretries once on failure before failing the batchtoken.rsresolves GitHub credentials via: explicit--github-token→gh auth token→ unauthenticatedrate_limiter.rsrespectsX-RateLimit-Remaining,X-RateLimit-Reset, andRetry-Afterheaders; pauses all in-flight requests on 429/403; disambiguates 403 permissions errors from rate limitingsend_with_rate_limit— one-retry layer for transport errors, 5xx, and rate-limit responses at the HTTP levelfind_assetDRY extraction — deduplicated asset lookup logic inGitHubClient-jflag usage, and examples added toapp-ampup.mdDownload Manager Tests
verify_artifact_with_empty_data_returns_empty_artifact_error— rejects zero-byte downloadsset_executable_permissions_with_multiple_files_sets_0o755_on_all— marks all files executable on Unixbackup_swap_with_existing_version_dir_replaces_contents— atomic replacement via backup-swap patternbackup_swap_with_stale_backup_dir_succeeds— handles leftover.olddir from crashed installappend_extension_with_semver_name_preserves_full_name— guards againstwith_extensiontruncating semver namesdownload_all_with_two_assets_writes_both_to_version_dir— both artifacts download successfullydownload_all_with_missing_asset_fails_without_partial_install— missing asset fails batch, no partial installdownload_all_with_sequential_mode_succeeds—-j 1sequential mode works correctly (Can be removed??)download_all_with_transient_failure_succeeds_on_retry— single 500 is retried and succeedsdownload_all_with_persistent_failure_fails_after_retry— persistent 500s exhaust retries, no partial installDeferred to follow-up PRs
[3/7] Installing components...); currently suppressedconfig.tomlsupport —[downloads] max_concurrent = 4config file optionBenchmark
-j 4(parallel): 45s vs-j 1(sequential): 69s — ~35% speedup with only 2 artifacts