Re-enable Docker Compose CLI E2E deployment tests - #19420
Re-enable Docker Compose CLI E2E deployment tests#19420Mitch Denny (mitchdenny) wants to merge 1 commit into
Conversation
Both tests in DockerDeploymentTests were disabled twice over: - [QuarantinedTest] was added in #15515 for issue #15511 — a release-branch package version split (SuppressFinalPackageVersion=true) that produced a CS1705 assembly version mismatch for Aspire.Hosting.Docker. That was a release-branch condition, not test flakiness. #15883 later relabelled the quarantine URLs to per-test failing-test issues #15882/#15871. - [ActiveIssue] for #15930 was added in #15931 because the tests installed the CLI behind an `if (isCI)` guard with no else branch, so outside a PR context they failed with `bash: aspire: command not found`. At that time these tests used CliE2ETestHelpers.CreateTestTerminal() — a bare (non-Docker) terminal — which is exactly the class of test #15930 describes. Both root causes are gone. The harness unification in #16131, #16298 and #16454 replaced the ad-hoc install block with CliInstallStrategy.Detect() + InstallAspireCliAsync(), which has an explicit CI fallback (InstallScript, quality=dev) and a local fallback (latest GA), and moved these tests onto CreateDockerTestTerminal(). VerifyPullRequestCliVersionAsync() also no-ops outside a PR context. Because eng/scripts/split-test-projects-for-ci.ps1 enumerates CI jobs with `--filter-not-trait quarantined=true`, a fully quarantined class produced no CI job at all. Removing both attributes restores the per-class job: the enumerated class list goes from 81 to 82 and now includes Aspire.Cli.EndToEnd.Tests.DockerDeploymentTests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 19420Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 19420" |
Tests selector (audit mode)The full test matrix and all jobs still run in audit mode. The tests and jobs below are what selective CI would run under enforcement. 1 / 101 test projects · 2 jobs, from 1 changed file. Selected test projects (1 / 101)
Selected jobs (2)
How these were chosen — grouped by what changed🧪 Job reasons
Selection computed for commit |
There was a problem hiding this comment.
Pull request overview
Re-enables Docker Compose CLI deployment E2E coverage after install-harness fixes made the disabling attributes obsolete.
Changes:
- Removes
ActiveIssueandQuarantinedTestfrom both Docker deployment tests. - Removes the unused test-utilities import.
Show a summary per file
| File | Description |
|---|---|
tests/Aspire.Cli.EndToEnd.Tests/DockerDeploymentTests.cs |
Restores both Docker Compose deployment tests to normal CI execution. |
Review details
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Balanced
| [Fact] | ||
| [ActiveIssue("https://github.com/microsoft/aspire/issues/15930")] | ||
| [QuarantinedTest("https://github.com/microsoft/aspire/issues/15882")] | ||
| public async Task CreateAndDeployToDockerCompose() |
|
Superseded by #19404, which will carry the attribute removal alongside the Docker Compose persistent-volume E2E coverage it unblocks. Keeping both in one PR avoids landing the re-enable and its first real consumer separately. The investigation and evidence recorded here still stand: the |
Description
Re-enables
DockerDeploymentTests.CreateAndDeployToDockerComposeandDockerDeploymentTests.CreateAndDeployToDockerComposeInteractive, which were disabled twice over ([ActiveIssue]and[QuarantinedTest]) and therefore ran in no workflow — not even the quarantine one.Root cause, per attribute
Neither disabling reason still applies.
1.
[QuarantinedTest]— originally #15511, via #15515. Not flakiness. On release branchesStabilizePackageVersion=truecreated a version split: stable packages resolved as13.2.0/ assembly13.2.0.0, whileSuppressFinalPackageVersion=truepackages such asAspire.Hosting.Dockerresolved as13.2.0-ci/ assembly42.42.42.42, producing a CS1705 assembly version mismatch. #15883 later relabelled the quarantine URLs to the per-test failing-test issues #15882 / #15871.2.
[ActiveIssue("#15930")]— via #15931. At the time, both tests installed the CLI behind anif (isCI)guard with noelsebranch, so outside a PR context nothing was installed and they failed ataspire newwithbash: aspire: command not found. That is exactly the captured signature in #15882 and #15871 — so those two issues are the same defect as #15930, not distinct Docker bugs.Worth noting: issue #15930's body states "Docker-based tests don't have this problem", yet lists these two as affected. That is not a contradiction — at the time of #15931 these tests called
CliE2ETestHelpers.CreateTestTerminal(), a bare (non-Docker) terminal. They only moved toCreateDockerTestTerminal()later, in the harness unification. TheActiveIssuewas correct when applied and is stale now.Why they pass now
The harness work in #16131, #16298 and #16454 replaced the ad-hoc install block with
CliInstallStrategy.Detect()+InstallAspireCliAsync().Detect()now ends in explicit fallbacks:That is precisely the missing
elsebranch #15930 described.VerifyPullRequestCliVersionAsync()also no-ops when there is no PR head SHA. The fix landed months ago; nobody removed the attributes.Why the class had no CI job
eng/scripts/split-test-projects-for-ci.ps1enumerates per-class jobs with--filter-not-trait quarantined=true, so a class whose tests are all quarantined disappears from the matrix entirely. RemovingActiveIssuealone would not have produced a job — both attributes had to go.Verified with CI's exact filter set:
DockerDeploymentTestspresentConfirmed in real CI on this PR: there are now 82
Cli.EndToEnd-*jobs andTests / Cli.EndToEnd-DockerDeploymentTests (ubuntu-latest)exists and passes.Evidence
CI (authoritative):
Cli.EndToEnd-DockerDeploymentTests— pass, 5m33s. The test step itself ran 2m38s, consistent with two real Compose deployments (an all-skipped class job returns in seconds under--ignore-exit-code 8). Whole-PR status: 363 pass, 0 fail.Local (macOS arm64, Docker 29.4.3,
ASPIRE_E2E_ARCHIVElocalhive build): these were quarantined for suspected flakiness, so the class was run 10 times, not once:No flakes, no retries, no assertion changes. Attributes were removed with
QuarantineTools(-u -m activeissue, then-u) perAGENTS.md, not by hand.Fixes #15930
Fixes #15882
Fixes #15871
Checklist