wip: set up very expensive tests to run in CI

[galargh]

Related Issues

Closes #12136

Proposed Changes

In this PR I set up a new workflow responsible for running very expensive tests. I also enable running expensive tests in our regular test workflow.

Additional Info

This is an updated version of #12234

In this iteration, the very expensive tests workflow is separate to the test workflow. This is so that we don't have to rerun all the tests on a label change.

The way the new workflow works is that when one adds a need/very-expensive-tests label to a PR, the workflow will start, remove the label, and execute the tests.

The new workflow will only execute the tests that have very expensive components to them. The information about that is currently stored in the ci cmd tool. It will be moved to the test files themselves in the future once we move more towards the self-identification setup.

This is not ready to be merged because the only very expensive test we have at the moment is failing with signal: killed. Here's a link to the most recent failure: https://github.com/filecoin-project/lotus/actions/runs/13706584703

Question: should the workflow comment on a PR with the very expensive test results? I think it might be useful since if one modifies labels after the very expensive tests are run, then the link to the very expensive tests run would disappear from the PR.

Checklist

Before you mark the PR ready for review, please make sure that:

• Commits have a clear commit message.
• PR title conforms with contribution conventions
• Update CHANGELOG.md or signal that this change does not need it per contribution conventions
• New features have usage guidelines and / or documentation updates in
  • Lotus Documentation
  • Discussion Tutorials
• Tests exist for new functionality or change in behavior
• CI is green

[BigLep]

Sweet - thanks for doing this!

the very expensive tests workflow is separate to the test workflow. This is so that we don't have to rerun all the tests on a label change.

Makes sense - sounds good.

remove the label

Why do we remove need/very-expensive-tests? I assume because this is the way to manually trigger the expensive test run for the PR.

I guess in a perfect world if the need/very-expensive-tests label was present, the expensive test groups would become a required CI check and would automatically get run whenever other tests are run.

The information about that is currently stored in the ci cmd tool. It will be moved to the test files themselves in the future once we move more towards the self-identification setup.

Sounds good.

Question: should the workflow comment on a PR with the very expensive test results? I think it might be useful since if one modifies labels after the very expensive tests are run, then the link to the very expensive tests run would disappear from the PR.

I think PR's labeled with need/very-expensive-tests should have the comment in the PR. (Ideally it would behave the same as every other test, except this PR just has extra tests being run as a result of the label.).

For a PR, I don't think we should be cutting issues if very expensive tests fail. I think we should only cut an issue if the very expensive cron job fails.

[BigLep]

Suggested change

	uses: ./.github/workflows/reusable-test.yml
	uses: ./.github/workflows/reusable-test.yml
	with:
	run_very_expensive_tests: false

I know that's the default, but it makes it explicit that this workflow isn't running with expensive tests.

[BigLep]

Suggested change

	Usage: "Whether the groups not containing any very expensive tests should be filtered out",
	Usage: "Whether to only include the groups with very expensive tests",

[BigLep]

Does this mean that if the boolean flag is not set that our test run includes all tests (i.e., not-very-expensive-tests + very-expensive-tests)?