Repository navigation
ci(zbugs-swift): warm the app up before the screenshot tests - #204
Merged
Merged
Conversation
Every failure of testListPagesPastTheFirst200 since Nov 3 was the first UI test after the simulator booted: a back tap lost, a launch query or a screenshot request timed out. Its launch took 9-79 s on CI against 3-12 s for testScreens on the warm simulator, which hasn't failed, and XCTest answers queries on the app's main thread, which the first sync keeps busy. The job now launches the app once after the boot and waits until it has taken in its first sync and gone quiet, then the tests run on the same build. On iPhone the test also taps back again if the filters haven't appeared and the back button is still there. Amp-Thread-ID: https://ampcode.com/threads/T-fbae32eb-2a12-4ebe-a716-b1b0bf7c246f Co-authored-by: tmm <tmm@tmm.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
testListPagesPastTheFirst200fails about one zbugs-screenshots run in seven, each time in a different way. A back tap doesn't navigate ("No the Status picker"), a launch query times out, or the final screenshot times out.Cause. All 6 failures since Nov 3 (5 on iPhone, 1 on iPad) happened in the first UI test after the simulator booted. Tests run in alphabetical order, so that is always this test. Its app launch took 9–79 s on CI, against 3–12 s for
testScreenson the now-warm simulator, andtestScreenshas never failed. A diagnostic run showed the runner I/O-starved (load 100–450) while the simulator's first-boot daemons were still busy. XCTest evaluates element queries on the app's main thread, which the first sync keeps busy, so queries, taps, and screenshot requests stall. On iOS 26, a back tap that is synthesized slowly enough becomes a long press, which opens the back-history menu instead of navigating. CI logged about 20 s between "Find" and "Synthesize" on the lost tap, so that is the likely mechanism, though it isn't proven.Fix.
issuePreloadV2and the app's CPU has gone quiet, then terminates the app. The tests then run on the warm simulator, reusing the same derived data.Checks. On a Mac (macos-26, Xcode 26.3), the extracted step run on an erased simulator warmed up in 11 s, and both tests passed. CI-load repeat results are below.