Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: buildless ecosystem ci #18525

Open
wants to merge 57 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
93196f0
init
Aslemammad Oct 30, 2024
582d059
debug
Aslemammad Oct 30, 2024
406b2da
debug
Aslemammad Oct 30, 2024
537d9f1
debug
Aslemammad Oct 30, 2024
fa83bca
debug
Aslemammad Oct 30, 2024
7c88c4c
debug
Aslemammad Oct 30, 2024
88cd5f5
debug
Aslemammad Oct 30, 2024
40f55e1
debug
Aslemammad Oct 31, 2024
abec9fb
trigger ci
sapphi-red Oct 31, 2024
f067ea6
debug
Aslemammad Oct 31, 2024
de78f9f
Update .github/workflows/ecosystem-ci-trigger.yml
Aslemammad Oct 31, 2024
560222a
debug
Aslemammad Oct 31, 2024
596b846
debug
Aslemammad Oct 31, 2024
de824cb
debug
Aslemammad Oct 31, 2024
ac51d2b
debug
Aslemammad Oct 31, 2024
b7c60df
debug
Aslemammad Oct 31, 2024
50dd0f4
trigger ci
sapphi-red Nov 1, 2024
399ce9d
debug
Aslemammad Nov 1, 2024
cdce56b
debug
Aslemammad Nov 1, 2024
444fc3a
debug
Aslemammad Nov 1, 2024
b48d3f4
debug
Aslemammad Nov 1, 2024
51e2efc
debug
Aslemammad Nov 1, 2024
a1fd622
debug
Aslemammad Nov 1, 2024
d560a03
debug
Aslemammad Nov 1, 2024
486135e
debug
Aslemammad Nov 1, 2024
a1ddb07
debug
Aslemammad Nov 1, 2024
e1fcdf9
debug
Aslemammad Nov 1, 2024
d841324
debug
Aslemammad Nov 1, 2024
ba291ed
debug
Aslemammad Nov 5, 2024
4ecbede
debug
Aslemammad Nov 5, 2024
b60a576
separation
Aslemammad Nov 5, 2024
66ee1a4
separation
Aslemammad Nov 5, 2024
396d523
separation
Aslemammad Nov 5, 2024
b1cfd85
back to basics
Aslemammad Nov 5, 2024
62faf78
finalize
Aslemammad Nov 5, 2024
c97f0eb
better way to remove reactions
Aslemammad Nov 5, 2024
5406307
revert preview-release.yml
Aslemammad Nov 5, 2024
ec00b0a
Update .github/workflows/ecosystem-ci-trigger.yml
Aslemammad Nov 6, 2024
4edf2c4
Update .github/workflows/ecosystem-ci-trigger.yml
Aslemammad Nov 6, 2024
00c68c6
Update .github/workflows/ecosystem-ci-trigger.yml
Aslemammad Nov 6, 2024
5f96097
Update .github/workflows/ecosystem-ci-trigger.yml
Aslemammad Nov 6, 2024
5395d82
Update .github/workflows/ecosystem-ci-trigger.yml
Aslemammad Nov 6, 2024
8127bfc
Update .github/workflows/ecosystem-ci-trigger.yml
Aslemammad Nov 6, 2024
6dfc0f9
move permissions check to the top
Aslemammad Nov 6, 2024
9c747f1
debug git show
Aslemammad Nov 9, 2024
1e46ed6
debug git show
Aslemammad Nov 9, 2024
f52e5e9
debug git show
Aslemammad Nov 9, 2024
d7bedb8
debug git show
Aslemammad Nov 9, 2024
1ff8a14
debug git show
Aslemammad Nov 9, 2024
2101400
debug git show
Aslemammad Nov 9, 2024
a32b13f
debug git show
Aslemammad Nov 9, 2024
5c07c4c
debug git show
Aslemammad Nov 9, 2024
c65eb11
debug git show
Aslemammad Nov 9, 2024
73708df
debug git show
Aslemammad Nov 9, 2024
eb3e33b
debug git show
Aslemammad Nov 9, 2024
0183d18
debug git show
Aslemammad Nov 9, 2024
7615586
find collisions properly
Aslemammad Nov 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 157 additions & 22 deletions .github/workflows/ecosystem-ci-trigger.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,51 @@ jobs:
runs-on: ubuntu-latest
if: github.repository == 'vitejs/vite' && github.event.issue.pull_request && startsWith(github.event.comment.body, '/ecosystem-ci run')
steps:
- uses: actions/github-script@v7
- name: Get PR Data
uses: actions/github-script@v7
id: get-pr-data
with:
script: |
console.log(`Get PR info: ${context.repo.owner}/${context.repo.repo}#${context.issue.number}`)
const { data: pr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number
})
core.setOutput('head_sha', pr.head.sha)
return {
num: context.issue.number,
branchName: pr.head.ref,
commit: pr.head.sha,
repo: pr.head.repo.full_name
}

- name: Check Package Existence
uses: actions/github-script@v7
id: check-package
with:
script: |
const prData = ${{ steps.get-pr-data.outputs.result }}
const url = `https://pkg.pr.new/vite@${prData.commit}`
const response = await fetch(url)
Copy link
Contributor Author

@Aslemammad Aslemammad Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not yet implement the HEAD for pkg.pr.new! So here instead of consuming the request body and waiting for it, we just check the status and then the step would be done.

So if it was fetching, it'd be aborted I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@dominikg dominikg Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you await fetch(), the full file has been downloaded already, its just not been parsed yet. of course if pkg.pr.new does not implement head we have to go with full, but should be considered as an optimization. or do you implement the endpoints about package information that the public registry has?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you await fetch(), the full file has been downloaded already

I did not know that, does not that defeat the purpose of streams (e.g. .body)?

As far as I know the await on .body would wait until finishing the fetch and meanwhile give us the result buffer by buffer while fetching.

but should be considered as an optimization

💯

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, maybe need to get more coffee and you are right.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha, thanks, let me get my own coffee too now 😅

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not yet implement the HEAD for pkg.pr.new!

Any plan to support it? It sounds interesting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any plan to support it? It sounds interesting.

I'll create an issue for it then!

console.log(`Package check URL: ${url}, Status: ${response.status}`)

// Add 'rocket' reaction to the issue comment
if (response.status === 404) {
const reaction = await github.rest.reactions.createForIssueComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: context.payload.comment.id,
content: 'rocket',
})
return { exists: false, reaction: reaction.id }
}

return { exists: true, reaction: null }
Aslemammad marked this conversation as resolved.
Show resolved Hide resolved

- name: Check User Permissions
uses: actions/github-script@v7
id: check-permissions
with:
script: |
const user = context.payload.sender.login
Expand All @@ -22,57 +66,147 @@ jobs:
repo: context.repo.repo,
username: user,
});
hasTriagePermission = data.user.permissions.triage
hasTriagePermission = ['triage', 'write', 'admin'].some(p => data.user.permissions[p]);
} catch (e) {
console.warn(e)
}

if (hasTriagePermission) {
console.log('Allowed')
if (allowed) {
console.log('User is allowed. Adding +1 reaction.')
await github.rest.reactions.createForIssueComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: context.payload.comment.id,
content: '+1',
})
} else {
console.log('Not allowed')
console.log('User is not allowed. Adding -1 reaction.')
await github.rest.reactions.createForIssueComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: context.payload.comment.id,
content: '-1',
})
throw new Error('not allowed')
throw new Error('User does not have the necessary permissions.')
}
- uses: actions/github-script@v7
id: get-pr-data

- name: Generate Token
id: generate-token
uses: tibdex/github-app-token@v2
with:
app_id: ${{ secrets.ECOSYSTEM_CI_GITHUB_APP_ID }}
installation_retrieval_payload: "${{ github.repository_owner }}/vite-ecosystem-ci"
private_key: ${{ secrets.ECOSYSTEM_CI_GITHUB_APP_PRIVATE_KEY }}

- name: Trigger Preview Release (if Package Not Found)
if: steps.check-package.outputs.exists == false
uses: actions/github-script@v7
id: trigger-preview-release
with:
github-token: ${{ steps.generate-token.outputs.token }}
script: |
console.log(`Get PR info: ${context.repo.owner}/${context.repo.repo}#${context.issue.number}`)
const { data: pr } = await github.rest.pulls.get({
const prData = ${{ steps.get-pr-data.outputs.result }}
console.log('Package not found, triggering preview release...')

// Add label "trigger: preview" to the PR
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number
issue_number: prData.num,
labels: ['trigger: preview']
})
return {
num: context.issue.number,
branchName: pr.head.ref,
repo: pr.head.repo.full_name
console.log('Added "trigger: preview" label.')

- name: Wait for Preview Release Completion (if Package Not Found)
if: steps.check-package.outputs.exists == false
uses: actions/github-script@v7
id: wait-preview-release
with:
script: |
const prData = ${{ steps.get-pr-data.outputs.result }}
const reaction = ${{ steps.check-package.outputs.reaction }}
const workflowFileName = 'preview-release.yml'
const workflow = await github.rest.actions.getWorkflow({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: workflowFileName,
})
const workflowId = workflow.data.id
console.log(`Waiting for workflow ID ${workflowId} to complete...`)

const maxRetries = 60 // Wait up to 10 minutes
const delay = 10000 // 10 seconds
let completed = false

for (let i = 0; i < maxRetries; i++) {
const runsData = await github.rest.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: workflowId,
head_sha: prData.commit,
per_page: 100,
page: 1,
})

const runs = runsData.data.workflow_runs

if (runs.length > 0) {
const latestRun = runs[0]
console.log(`Latest run status: ${latestRun.status}, conclusion: ${latestRun.conclusion}`)
if (latestRun.status === 'completed') {
if (latestRun.conclusion === 'success') {
console.log('Preview release workflow completed successfully.')
completed = true
break
} else {
throw new Error('Preview Release workflow failed.')
}
}
}

console.log(`Retrying... (${i + 1}/${maxRetries})`)
await new Promise(resolve => setTimeout(resolve, delay))
}
- id: generate-token
uses: tibdex/github-app-token@v2

if (!completed) {
throw new Error('Preview Release workflow did not complete in time.')
}

// Remove the 'rocket' reaction
if (reaction) {
await github.rest.reactions.deleteForIssueComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: context.payload.comment.id,
reaction_id: reaction,
})
console.log('Removed "rocket" reaction.')
}

- name: Checkout
uses: actions/checkout@v4
with:
app_id: ${{ secrets.ECOSYSTEM_CI_GITHUB_APP_ID }}
installation_retrieval_payload: "${{ github.repository_owner }}/vite-ecosystem-ci"
private_key: ${{ secrets.ECOSYSTEM_CI_GITHUB_APP_PRIVATE_KEY }}
- uses: actions/github-script@v7
fetch-depth: 0

- name: Check Commit Hash Ambiguity
id: check_ambiguity
run: |
HEAD_SHA=${{ steps.get-pr-data.outputs.head_sha }}
COMMIT_SHORT=${HEAD_SHA:0:7}

if git show "$COMMIT_SHORT"; then
echo "COLLISION=false" >> $GITHUB_ENV
else
echo "COLLISION=true" >> $GITHUB_ENV
fi

- name: Trigger Downstream Workflow
uses: actions/github-script@v7
id: trigger
env:
COMMENT: ${{ github.event.comment.body }}
with:
github-token: ${{ steps.generate-token.outputs.token }}
result-encoding: string
script: |
const comment = process.env.COMMENT.trim()
const prData = ${{ steps.get-pr-data.outputs.result }}
Expand All @@ -88,6 +222,7 @@ jobs:
prNumber: '' + prData.num,
branchName: prData.branchName,
repo: prData.repo,
commit: process.env.COLLISION === 'false' ? prData.commit : '',
suite: suite === '' ? '-' : suite
}
})