Make Release EA Process per-adapter #4133

cawthorne · 2025-10-21T14:59:35Z

Enable Per-Adapter EA Releases

⚠️ DRAFT - Untested. Needs validation before production use.

Problem

EA releases are all-or-none. If one adapter is ready, ALL changed adapters must be released together.

Solution

Enable releasing adapters individually or all together (your choice).

What Changed

1. `deploy.yml` (1 line added)

Line 166: Enable per-adapter PRs in infra-k8s

-F per-adapter-prs=true

2. `release.yml` (Complete rewrite)

Selective releases with security validation:

Default: Release all pending (gh workflow run release.yml)
Optional: Release specific (gh workflow run release.yml -f adapters=coingecko)
Security: Validates adapter exists in MASTERLIST.md, version matches, commit SHA verified, image digest tracked
Image digest: Releases by immutable SHA256 (not mutable tags)
GitHub releases: Creates individual release per adapter

3. `upsert-release-pr.yml` (1 step added)

Auto-populates RELEASES.yaml:

Parses MASTERLIST.md changes → adds to RELEASES.yaml (tested_in_infra: false)
Note: Infra testing status updated by webhook from infra-k8s (see companion PR)

4. `RELEASES.yaml` (NEW - 135 lines)

Central tracking file:

pending:
  coingecko:
    version: "1.5.0"
    built_from_commit: "abc123..."
    image_digest: "sha256:789..."
    tested_in_infra: true
released:
  tiingo:
    version: "1.2.3"

Has comprehensive inline docs (all instructions in the file itself).

5. `scripts/release-manager.sh` (NEW - 345 lines)

CLI tool (mostly automated, rarely needed):

./scripts/release-manager.sh list    # Check what's pending
./scripts/release-manager.sh status coingecko

How It Works Together

Full Flow

Code merged → upsert-release-pr.yml creates Release PR
Adds adapter to RELEASES.yaml (tested_in_infra: false)
deploy.yml triggers infra-k8s with per-adapter-prs=true
infra-k8s creates separate PR per adapter (branch: images/ea/<adapter>)
Team tests and merges infra PR
infra-k8s webhook fires immediately → Updates Release PR with tested status + image digest
Team triggers release when ready: gh workflow run release.yml (or merge Release PR for auto-release)
release.yml validates and publishes

Key Points

✅ No waiting for next EA merge - status updates immediately
✅ Release PR always shows current tested status
✅ Per-adapter testing and release isolation
✅ Uses existing GitHub token (no new infrastructure)

Security Validation

Every release checks:

Adapter exists in MASTERLIST.md (prevents fake adapters)
Version matches MASTERLIST.md (prevents version manipulation)
Commit SHA verified in repo history (prevents external code)
Image digest tracked from infra (prevents image substitution)

⚠️ Testing Status

NOT TESTED - Hasn't run in GitHub Actions or been verified with real data.

Before Production

Validate syntax: yamllint .github/workflows/*.yml
Test with non-critical adapter
Use dry-run mode first: gh workflow run release.yml -f dry-run=true
Monitor carefully
Fix issues as they arise

Rollback

Change line 166 in deploy.yml: -F per-adapter-prs=false

Files Modified

.github/workflows/deploy.yml (1 line)
.github/workflows/release.yml (rewritten)
.github/workflows/upsert-release-pr.yml (+1 step, note about webhook)
RELEASES.yaml (NEW)
scripts/release-manager.sh (NEW, needs chmod +x)

Requires: Companion PR in infra-k8s (merge both together)

Infrastructure: Uses existing GitHub token from infra-k8s (no new setup needed)

changeset-bot · 2025-10-21T14:59:40Z

⚠️ No Changeset found

Latest commit: 73a9b92

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

.github/workflows/release.yml

+    name: Determine Release Mode
+    runs-on: ubuntu-latest
+    outputs:
+      release-mode: ${{ steps.determine.outputs.release-mode }}
+      adapter-list: ${{ steps.determine.outputs.adapter-list }}
+    steps:
+      - uses: actions/checkout@v5
+        with:
+          persist-credentials: false
+          fetch-depth: 2
+
+      - name: Determine release mode
+        id: determine
+        env:
+          EVENT_NAME: ${{ github.event_name }}
+          ADAPTERS_INPUT: ${{ inputs.adapters }}
+          BUILD_ALL: ${{ inputs.build-all }}
+        run: |
+          if [ "$EVENT_NAME" = "workflow_dispatch" ] && [ -n "$ADAPTERS_INPUT" ]; then
+            echo "release-mode=selective" >> $GITHUB_OUTPUT
+            echo "Using selective release mode for: $ADAPTERS_INPUT"
+          else
+            echo "release-mode=legacy" >> $GITHUB_OUTPUT
+            echo "Using legacy release mode (MASTERLIST.md trigger or build-all)"
+          fi
+
+  # Selective release: Parse adapters from RELEASES.yaml
+  parse-selective-adapters:


The recommended fix is to add a permissions: key with the minimum necessary privileges. Typically, for release workflows that read contents but do not need to push code, at minimum you'll need contents: read. This can be set at the top of the workflow to apply to all jobs (the normal approach unless a job needs more/less permissions), or at the job level for the jobs that need custom permissions. Since there is no evidence in the provided snippet that any step requires write permissions, we'll set contents: read only (which will be secure). Add the following at the root of the workflow, just after the name: (e.g., line 2). No imports, package changes, or additional steps are needed; this is a YAML metadata change.

.github/workflows/release.yml

+    name: Compute changed adapters (Legacy)
+    needs: determine-release-mode
+    if: needs.determine-release-mode.outputs.release-mode == 'legacy'
    runs-on: [ubuntu-latest]
    env:
      BUILD_ALL: ${{ inputs.build-all }}
    outputs:
      adapter-list: ${{ steps.changed-adapters.outputs.CHANGED_ADAPTERS }}
    steps:
      - uses: actions/checkout@v5
        with:
          persist-credentials: false
          fetch-depth: 2
      - name: Set up and install dependencies
        uses: ./.github/actions/setup
        with:
          skip-setup: true
      - name: Build list of changed packages and changed adapters
        id: changed-adapters
        env:
          UPSTREAM_BRANCH: HEAD~1
        run: |
          ./.github/scripts/changed-adapters.sh

+  # Merge adapter lists from both modes
+  merge-adapter-lists:
+    name: Merge Adapter Lists
+    needs: [determine-release-mode, parse-selective-adapters, calculate-changes]
+    if: always() && !cancelled()
+    runs-on: ubuntu-latest
+    outputs:
+      adapter-list: ${{ steps.merge.outputs.adapter-list }}
+      release-mode: ${{ needs.determine-release-mode.outputs.release-mode }}
+    steps:
+      - name: Merge lists
+        id: merge
+        env:
+          RELEASE_MODE: ${{ needs.determine-release-mode.outputs.release-mode }}
+          SELECTIVE_LIST: ${{ needs.parse-selective-adapters.outputs.adapter-list }}
+          LEGACY_LIST: ${{ needs.calculate-changes.outputs.adapter-list }}


To resolve the issue, we need to explicitly declare a minimal permissions block for the calculate-changes job in .github/workflows/release.yml. The recommended least-privilege setting is typically contents: read, unless the job requires additional (write) permissions for specific tasks. In this job, it does not appear to need anything beyond read access to repository contents.

Specifically, add this block just under calculate-changes: (i.e., under line 206):

permissions: contents: read

No imports or other changes are required. This change is contained entirely to the .github/workflows/release.yml file and does not affect any other job or workflow functionality.

Make Release EA Process per-adapter

73a9b92

github-advanced-security bot found potential problems Oct 21, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Make Release EA Process per-adapter #4133

Make Release EA Process per-adapter #4133

Uh oh!

cawthorne commented Oct 21, 2025

Uh oh!

changeset-bot bot commented Oct 21, 2025

Uh oh!

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

@@ -1,4 +1,6 @@
             name: Release
+            permissions:
+              contents: read
             on:
               push:

Uh oh!

Make Release EA Process per-adapter #4133

Are you sure you want to change the base?

Make Release EA Process per-adapter #4133

Uh oh!

Conversation

cawthorne commented Oct 21, 2025

Enable Per-Adapter EA Releases

Problem

Solution

What Changed

1. deploy.yml (1 line added)

2. release.yml (Complete rewrite)

3. upsert-release-pr.yml (1 step added)

4. RELEASES.yaml (NEW - 135 lines)

5. scripts/release-manager.sh (NEW - 345 lines)

How It Works Together

Full Flow

Key Points

Security Validation

⚠️ Testing Status

Before Production

Rollback

Files Modified

Uh oh!

changeset-bot bot commented Oct 21, 2025

⚠️ No Changeset found

Uh oh!

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

1. `deploy.yml` (1 line added)

2. `release.yml` (Complete rewrite)

3. `upsert-release-pr.yml` (1 step added)

4. `RELEASES.yaml` (NEW - 135 lines)

5. `scripts/release-manager.sh` (NEW - 345 lines)