Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(v3): test and lint v-next packages lazily #5564

Merged
merged 5 commits into from
Aug 5, 2024

Conversation

galargh
Copy link
Member

@galargh galargh commented Aug 1, 2024

  • Because this PR includes a bug fix, relevant tests have been included.
  • Because this PR includes a new feature, the change was previously discussed on an Issue or with someone from the team.
  • I didn't do anything of this.

This PR is a subset of #5301 and, unlike the original, it targets the v-next branch.

In this PR, I introduce the notion of deciding what packages to build in CI lazily. The intention is to build and test only these packages that were themselves modified or if one of their local dependencies (including transitive dependencies) were modified.

To achieve this, I added a script which parses the pnpm lockfile, builds up a map of local transitive depencies out of it, and outputs a filter that can be used directly with https://github.com/dorny/paths-filter (an action that filters a list of packages based on the modified paths).

The output of the script I added can be controlled with 2 environment variables: PACKAGE_IGNORE (to filter out the packages that we don't care about) and COMMON_FILTERS (to configure glob paths which modification should result in building all the packages).

Example

In our script execution, we want to exclude the root package, the v2 packages and the example-project so we set PACKAGE_IGNORE=["packages/", ".", "v-next/example-project"].

We also want to ensure that modifying anything under config-v-next builds all of the packages so we set COMMON_FILTERS=["config-v-next/**"].

The output of our script (after parsing it with `yq`) would be:
core:
  - config-v-next/**
  - v-next/core/**
  - v-next/hardhat-errors/**
  - v-next/hardhat-utils/**
  - v-next/hardhat-node-test-reporter/**
  - v-next/hardhat-test-utils/**
hardhat:
  - config-v-next/**
  - v-next/hardhat/**
  - v-next/core/**
  - v-next/hardhat-errors/**
  - v-next/hardhat-utils/**
  - v-next/hardhat-zod-utils/**
  - v-next/hardhat-node-test-reporter/**
  - v-next/hardhat-test-utils/**
hardhat-build-system:
  - config-v-next/**
  - v-next/hardhat-build-system/**
  - v-next/hardhat-errors/**
  - v-next/hardhat-utils/**
  - v-next/hardhat-node-test-reporter/**
  - v-next/hardhat-test-utils/**
hardhat-errors:
  - config-v-next/**
  - v-next/hardhat-errors/**
  - v-next/hardhat-utils/**
  - v-next/hardhat-node-test-reporter/**
hardhat-node-test-reporter:
  - config-v-next/**
  - v-next/hardhat-node-test-reporter/**
hardhat-test-utils:
  - config-v-next/**
  - v-next/hardhat-test-utils/**
  - v-next/hardhat-errors/**
  - v-next/hardhat-utils/**
  - v-next/hardhat-node-test-reporter/**
hardhat-utils:
  - config-v-next/**
  - v-next/hardhat-utils/**
  - v-next/hardhat-node-test-reporter/**
hardhat-zod-utils:
  - config-v-next/**
  - v-next/hardhat-zod-utils/**
  - v-next/core/**
  - v-next/hardhat-node-test-reporter/**
  - v-next/hardhat-test-utils/**
  - v-next/hardhat-errors/**
  - v-next/hardhat-utils/**
template-package:
  - config-v-next/**
  - v-next/template-package/**
  - v-next/hardhat-node-test-reporter/**
  - v-next/hardhat-test-utils/**
  - v-next/hardhat-errors/**
  - v-next/hardhat-utils/**

Now, let's say we modify v-next/hardhat-zod-utils/package.json. If we, then, used the script output as the input to dorny/paths-filter action, we would learn that we should build only the following packages:

  • hardhat
  • hardhat-zod-utils

Copy link

vercel bot commented Aug 1, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
hardhat ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 2, 2024 4:10pm

Copy link

changeset-bot bot commented Aug 1, 2024

⚠️ No Changeset found

Latest commit: f8dc86c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@galargh galargh force-pushed the galargh/github-actions-laziness branch from 7f164e9 to 0cc2836 Compare August 2, 2024 11:24
@galargh galargh changed the title ci: test and lint v-next packages lazily ci: test and lint v-next packages lazily (v3) Aug 2, 2024
@galargh galargh changed the title ci: test and lint v-next packages lazily (v3) ci(v3): test and lint v-next packages lazily Aug 2, 2024
@galargh galargh added the no changeset needed This PR doesn't require a changeset label Aug 2, 2024
@galargh galargh requested a review from alcuadrado August 2, 2024 11:56
@galargh galargh marked this pull request as ready for review August 2, 2024 11:56
Copy link
Member

@alcuadrado alcuadrado left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a lockfile is modified we should run everything

@galargh
Copy link
Member Author

galargh commented Aug 2, 2024

I added pnpm lockfile handling in f8dc86c

@galargh galargh requested a review from alcuadrado August 2, 2024 16:16
@galargh galargh merged commit 4b07993 into v-next Aug 5, 2024
52 checks passed
@galargh galargh deleted the galargh/github-actions-laziness branch August 5, 2024 09:08
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
no changeset needed This PR doesn't require a changeset status:ready This issue is ready to be worked on
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants