Skip to content

Conversation

@pietern
Copy link
Contributor

@pietern pietern commented Nov 28, 2025

This PR triggers the test-exp-apps-mcp Makefile target to verify it runs correctly.

@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Nov 28, 2025

Commit: c145f24

Run: 19757173200

Env 🟨​KNOWN 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 2 371 621 19:18
🟨​ aws windows 7 2 373 619 20:37
💚​ aws-ucws linux 7 2 514 506 33:15
💚​ aws-ucws windows 7 2 516 504 33:39
💚​ azure linux 1 4 371 620 20:35
💚​ azure windows 1 4 373 618 19:43
💚​ azure-ucws linux 1 4 510 505 31:01
💚​ azure-ucws windows 1 4 512 503 31:26
💚​ gcp linux 1 4 364 624 24:07
💚​ gcp windows 1 4 366 622 22:33
9 failing tests:
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/run/app-with-job 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
Top 35 slowest tests (at least 2 minutes):
duration env testname
5:41 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:40 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:30 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:24 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:17 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:13 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:13 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:12 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:11 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:57 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
4:08 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
3:18 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic
3:14 azure-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
3:04 aws-ucws windows TestAccept/bundle/resources/experiments/basic/DATABRICKS_BUNDLE_ENGINE=direct
2:58 azure-ucws linux TestAccept/bundle/resources/synced_database_tables/basic
2:48 gcp linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9
2:47 aws-ucws linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9
2:37 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
2:33 aws-ucws linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.12
2:25 aws-ucws linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.10
2:23 azure-ucws linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.13
2:23 azure-ucws linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.12
2:21 aws-ucws windows TestAccept/bundle/resources/experiments/basic/DATABRICKS_BUNDLE_ENGINE=terraform
2:19 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
2:16 gcp linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.11
2:15 gcp linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.11
2:13 gcp linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.13
2:12 aws-ucws linux TestAccept/bundle/resources/registered_models/basic/DATABRICKS_BUNDLE_ENGINE=terraform
2:12 azure-ucws linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.13
2:10 azure-ucws windows TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.10
2:10 azure-ucws linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.10
2:09 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
2:05 gcp linux TestAccept/bundle/deployment/bind/job/job-spark-python-task
2:04 azure-ucws linux TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9
2:03 gcp windows TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.12

github-merge-queue bot pushed a commit that referenced this pull request Dec 2, 2025
## Changes

A new tool `testmask` under `tools/` that determines the set of Makefile
targets to run for testing.

The test jobs all depend on a job that runs this tool and use its output
to determine if they should run or skip. If a change only touches files
under `experimental/ssh`, for example, it won't trigger the main test
target. We can extend this to add more selective targets, only if we are
sure that there is no dependency between the files matched for the
target and other files in the repository. This, for now, relies on
convention rather than strict enforcement.

### How

Every file in the diff between the PR and the PR base is attributed to
one of the targets. If all files map to the same target, only that
target is run. If files map to multiple targets, multiple targets are
run. If there are files that cannot be mapped to a specific target, then
the main `test` target is run.

### Mapping to GitHub Actions

This approach uses a GitHub Actions test job for every target. This
results in a status check mark for each of them, making it easy to
eyeball which tests ran on a PR, which tests can on main, and when they
broke. Executing the same locally is straightforward because they are
all named after Makefile targets.

It applies to pull requests and merge queue checks. Scheduled runs and
main branch pushes always run the full test suite.

## Why

Not all tests need to run for every change. Being more selective reduces
build times and removes noise in PRs.

## Tests

The tests for this PR show it works.

A new test in `tools/testmask` confirms that the targets it maps to are
present in the Makefile.

PRs for each of the targets:
* #4023 
* #4024
* #4025
* #4026
Base automatically changed from testmask to main December 2, 2025 14:33
@pietern pietern closed this Dec 2, 2025
@pietern pietern deleted the test-exp-apps-mcp branch December 2, 2025 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants