test: add tart VM lifecycle integration tests by STRRL · Pull Request #55 · boring-design/elastic-fruit-runner

STRRL · 2026-04-07T05:48:34Z

Summary

Add BDD integration tests for tart VM lifecycle (pull, clone, start, IP, exec, stop, delete, cleanup)
Uses ghcr.io/cirruslabs/macos-tahoe-base:latest as smaller test image (no Xcode)
Tart steps auto-skip when tart binary is not available
CI matrix adds "Tart" job on self-hosted macOS arm64 runner with nested virtualization

Test plan

Verify tart tests pass on self-hosted macOS arm64 runner
Verify non-tart integration tests still pass
Verify tart tests skip gracefully on runners without tart installed

cloudflare-workers-and-pages · 2026-04-07T05:48:55Z

Deploying elastic-fruit-runner with Cloudflare Pages

Latest commit:	`90f9e15`
Status:	✅ Deploy successful!
Preview URL:	https://571fd2af.elastic-fruit-runner.pages.dev
Branch Preview URL:	https://feat-tart-integration-tests.elastic-fruit-runner.pages.dev

View logs

Add BDD scenarios for tart manager operations with nested virtualization: - Pull, clone, start, IP discovery, SSH exec, stop, delete - List and cleanup orphaned VMs with prefix matching Uses ghcr.io/cirruslabs/macos-tahoe-base:latest as a smaller test image (configurable via EFR_TEST_TART_IMAGE). Tart steps auto-skip via godog.ErrPending when tart binary is not available. CI matrix gains a "Tart" job running on self-hosted macOS arm64 runner with nested virtualization enabled.

chatgpt-codex-connector · 2026-04-07T05:52:34Z

💡 Codex Review

elastic-fruit-runner/.github/workflows/controller-integration-test.yaml

Line 16 in 3a115bb

runs-on: [self-hosted, linux, arm64]

Run Tart integration matrix on a macOS runner

The integration matrix now runs entirely on [self-hosted, linux, arm64], including the new Tart case, so the Tart lifecycle scenario is not actually exercised on the intended platform. In this repo Tart is the macOS VM path (README), and the new step a tart manager explicitly marks the scenario pending when tart is unavailable (steps_test.go), which means Linux runners can report a passing/green job without validating Tart behavior. Use a macOS label for the Tart matrix entry (or matrix-specific runs-on) so this test provides real coverage.

elastic-fruit-runner/test/integration/steps_test.go

Lines 722 to 726 in 3a115bb

    
           sc.Step(`^I stop and delete the VM$`, func() error { 
        
           	if err := state.tartMgr.Stop(context.Background(), state.tartVMName); err != nil { 
        
           		return fmt.Errorf("stop VM: %w", err) 
        
           	} 
        
           	return state.tartMgr.Delete(context.Background(), state.tartVMName)

Ensure Tart VMs are torn down on step failure

VM cleanup is only performed in this explicit scenario step, so if any earlier Tart step fails (pull/clone/start/ip/exec), the remaining steps are skipped and stop/delete never runs. Because these tests run on persistent self-hosted machines, leaked VMs can accumulate and interfere with subsequent jobs. Add unconditional teardown in sc.After (or equivalent) that attempts to stop/delete state.tartVMName whenever it was created.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

codecov · 2026-04-07T05:55:08Z

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/api/server.go	66.66%	1 Missing ⚠️

Flag	Coverage Δ
integration-GitHub App	`42.50% <66.66%> (+5.07%)`	⬆️
integration-PAT	`41.21% <66.66%> (-0.72%)`	⬇️
integration-Tart	`?`
unittests	`20.42% <0.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
internal/management/jobs.go	`73.91% <ø> (ø)`
internal/api/server.go	`78.90% <66.66%> (ø)`

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Use per-matrix runs-on so PAT/GitHub App tests run on Linux arm64 while Tart tests run on macOS arm64 (required for Apple Virtualization.framework and nested virtualization).

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 45b7b78937

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-07T06:02:35Z

features/tart/vm_lifecycle.feature

+
+  Scenario: list and cleanup orphaned VMs
+    Given a tart manager
+    When I clone a VM with a random name


Pull image before cloning in orphan-cleanup scenario

The list and cleanup orphaned VMs scenario clones immediately without first ensuring the test image is present locally, so on a clean runner (or when this scenario is run in isolation) Clone can fail before any cleanup assertions run. This makes the scenario order-dependent on pull, clone, start, exec, and cleanup a VM instead of being self-contained.

Useful? React with 👍 / 👎.

… API GitHub Actions API sends job results in lowercase ("succeeded", "failed") but the mapping code expected Title Case ("Succeeded", "Failed"), causing all completed jobs to show as FAIL in the dashboard (issue #68).

STRRL · 2026-04-07T06:08:56Z

internal/management/jobs.go

+	"failed":    {},
 	"Succeeded": {},
 	"Failed":    {},
 }


😮‍💨 slop

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90f9e155d2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-07T06:12:04Z

test/integration/steps_test.go

+				_ = state.tartMgr.Stop(context.Background(), name)
+				_ = state.tartMgr.Delete(context.Background(), name)


Propagate tart cleanup failures instead of discarding them

This step ignores both Stop and Delete errors, so the scenario can still pass even when cleanup fails for prefixed VMs (especially stale orphans from prior runs). Because the next assertion only verifies the current cloned VM name, failures on other matched VMs are silently masked, which can leave orphaned Tart VMs behind and hide regressions in the cleanup path.

Useful? React with 👍 / 👎.

STRRL force-pushed the feat/tart-integration-tests branch from 3a115bb to 660b772 Compare April 7, 2026 05:52

STRRL added 2 commits April 6, 2026 22:55

ci: run tart integration tests on self-hosted macOS arm64 runner

e7ff8ca

Use per-matrix runs-on so PAT/GitHub App tests run on Linux arm64 while Tart tests run on macOS arm64 (required for Apple Virtualization.framework and nested virtualization).

ci: run all common features in both PAT and GitHub App jobs

45b7b78

chatgpt-codex-connector bot reviewed Apr 7, 2026

View reviewed changes

STRRL commented Apr 7, 2026

View reviewed changes

internal/management/jobs.go

"failed": {},

"Succeeded": {},

"Failed": {},

}

Copy link
Copy Markdown

Member Author

STRRL Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😮‍💨 slop

chatgpt-codex-connector bot reviewed Apr 7, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: add tart VM lifecycle integration tests#55

test: add tart VM lifecycle integration tests#55
STRRL wants to merge 4 commits intomainfrom
feat/tart-integration-tests

STRRL commented Apr 7, 2026

Uh oh!

cloudflare-workers-and-pages bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector bot commented Apr 7, 2026

Uh oh!

codecov bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Apr 7, 2026

Uh oh!

STRRL Apr 7, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		_ = state.tartMgr.Stop(context.Background(), name)
		_ = state.tartMgr.Delete(context.Background(), name)

Conversation

STRRL commented Apr 7, 2026

Summary

Test plan

Uh oh!

cloudflare-workers-and-pages bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying elastic-fruit-runner with Cloudflare Pages

Uh oh!

chatgpt-codex-connector bot commented Apr 7, 2026

💡 Codex Review

Uh oh!

codecov bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

STRRL Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

cloudflare-workers-and-pages bot commented Apr 7, 2026 •

edited

Loading

codecov bot commented Apr 7, 2026 •

edited

Loading