Skip to content

Comments

feat(ampup): bounded-concurrency parallel downloads#5

Open
mitchhs12 wants to merge 1 commit intomainfrom
mitchhs12/ampup-download-manager
Open

feat(ampup): bounded-concurrency parallel downloads#5
mitchhs12 wants to merge 1 commit intomainfrom
mitchhs12/ampup-download-manager

Conversation

@mitchhs12
Copy link
Contributor

@mitchhs12 mitchhs12 commented Feb 23, 2026

Summary

Implements bounded-concurrency parallel downloads for installing multiple components simultaneously, addressing edgeandnode/amp#1762.

  • DownloadManager — new module using Tokio Semaphore + JoinSet for parallel artifact downloads (default 4 concurrent, configurable via -j/--jobs)
  • Staging directory with atomic rename — downloads write to a temp staging dir (sibling of version dir); only after all downloads succeed is it atomically renamed to the final version directory. No partial installs
  • One retry per downloaddownload_with_retry retries once on failure before failing the batch
  • Fail-fast cancellation — if any download fails, all in-flight tasks are cancelled and staging is cleaned up
  • Token resolution chaintoken.rs resolves GitHub credentials via: explicit --github-tokengh auth token → unauthenticated
  • Rate limiterrate_limiter.rs respects X-RateLimit-Remaining, X-RateLimit-Reset, and Retry-After headers; pauses all in-flight requests on 429/403; disambiguates 403 permissions errors from rate limiting
  • send_with_rate_limit — one-retry layer for transport errors, 5xx, and rate-limit responses at the HTTP level
  • find_asset DRY extraction — deduplicated asset lookup logic in GitHubClient
  • Feature docs updated — installation flow, -j flag usage, and examples added to app-ampup.md

Download Manager Tests

  1. verify_artifact_with_empty_data_returns_empty_artifact_error — rejects zero-byte downloads
  2. set_executable_permissions_with_multiple_files_sets_0o755_on_all — marks all files executable on Unix
  3. backup_swap_with_existing_version_dir_replaces_contents — atomic replacement via backup-swap pattern
  4. backup_swap_with_stale_backup_dir_succeeds — handles leftover .old dir from crashed install
  5. append_extension_with_semver_name_preserves_full_name — guards against with_extension truncating semver names
  6. download_all_with_two_assets_writes_both_to_version_dir — both artifacts download successfully
  7. download_all_with_missing_asset_fails_without_partial_install — missing asset fails batch, no partial install
  8. download_all_with_sequential_mode_succeeds-j 1 sequential mode works correctly (Can be removed??)
  9. download_all_with_transient_failure_succeeds_on_retry — single 500 is retried and succeeds
  10. download_all_with_persistent_failure_fails_after_retry — persistent 500s exhaust retries, no partial install

Deferred to follow-up PRs

  • Aggregate progress reporting — visual download progress (e.g. [3/7] Installing components...); currently suppressed
  • Per-artifact checksum/attestation verification — currently only checks non-empty; checksum validation will be added once release assets include checksums
  • config.toml support[downloads] max_concurrent = 4 config file option

Benchmark

  • -j 4 (parallel): 45s vs -j 1 (sequential): 69s — ~35% speedup with only 2 artifacts

@mitchhs12 mitchhs12 force-pushed the mitchhs12/ampup-download-manager branch from 1bfd0cb to c58f15a Compare February 23, 2026 23:18
Introduce DownloadManager with Semaphore + JoinSet for parallel artifact
downloads (default 4, configurable via -j/--jobs). Includes staging
directory with atomic rename for no-partial-installs, one retry per
download, fail-fast cancellation, token resolution chain (explicit →
gh auth token → unauthenticated), and shared rate limiter with 429/403
handling.
@mitchhs12 mitchhs12 force-pushed the mitchhs12/ampup-download-manager branch from c58f15a to 130eef3 Compare February 23, 2026 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant