Fix per-test isolation flakiness: relaunch instead of UI logout; 4 workers
The first full 8-worker run surfaced 52 failures, 28 of them "Failed to log out" (UITestHelpers:86) — forcing a profile-navigation logout between every test (each test = new account) is fragile, and 8 parallel simulator clones thrashed the machine (the remaining failures were UI timeouts under that load). - AuthenticatedUITestCase: relaunchBetweenTests = true. A fresh app launch with --reset-state lands on the login screen, so each test logs in as its own account with NO UI logout between tests. Removed the ensureLoggedOut call. - run_ui_tests.sh: default workers 8 -> 4 (reliable on a Mac mini; each test now relaunches + creates an account, so the bottleneck is CPU/simulator). Verified: ContractorUITests (was ~15 logout failures) now passes at 4 workers, 0 leaked accounts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -17,6 +17,12 @@ class AuthenticatedUITestCase: BaseUITestCase {
|
|||||||
/// (sets OnboardingState.hasCompletedOnboarding) so login lands on main tabs.
|
/// (sets OnboardingState.hasCompletedOnboarding) so login lands on main tabs.
|
||||||
override var completeOnboarding: Bool { true }
|
override var completeOnboarding: Bool { true }
|
||||||
|
|
||||||
|
/// Per-test isolation relaunches the app fresh for every test. With
|
||||||
|
/// --reset-state this lands on the login screen, so each test logs in as its
|
||||||
|
/// own fresh account WITHOUT a fragile UI logout between tests (the old
|
||||||
|
/// logout-via-profile path was the #1 source of flakes under load).
|
||||||
|
override var relaunchBetweenTests: Bool { true }
|
||||||
|
|
||||||
/// Credentials for the Kratos APP identity used to seed data over the API.
|
/// Credentials for the Kratos APP identity used to seed data over the API.
|
||||||
///
|
///
|
||||||
/// ⚠️ TWO DIFFERENT "admin@honeydue.com" EXIST — do NOT "fix" Test1234 to password123:
|
/// ⚠️ TWO DIFFERENT "admin@honeydue.com" EXIST — do NOT "fix" Test1234 to password123:
|
||||||
@@ -116,11 +122,10 @@ class AuthenticatedUITestCase: BaseUITestCase {
|
|||||||
try super.setUpWithError()
|
try super.setUpWithError()
|
||||||
|
|
||||||
if usesFreshAccount {
|
if usesFreshAccount {
|
||||||
// Per-test isolation: every test logs in as its OWN fresh, pre-verified
|
// Per-test isolation: relaunchBetweenTests gives every test a fresh
|
||||||
// account, seeds under its token, and deletes it in teardown. The app
|
// app launch that lands on the login screen (--reset-state), so we
|
||||||
// may be reused from a previous test (still logged in as that test's
|
// log in as a brand-new pre-verified account, seed under its token,
|
||||||
// account), so always log out first.
|
// and delete it in teardown. No UI logout between tests.
|
||||||
UITestHelpers.ensureLoggedOut(app: app)
|
|
||||||
let acct = TestAccount.create(domain: accountDomain)
|
let acct = TestAccount.create(domain: accountDomain)
|
||||||
account = acct
|
account = acct
|
||||||
session = acct.session
|
session = acct.session
|
||||||
|
|||||||
@@ -34,7 +34,10 @@ SCHEME="HoneyDueUITests"
|
|||||||
API_SCHEME="HoneyDueAPITests"
|
API_SCHEME="HoneyDueAPITests"
|
||||||
TARGET="HoneyDueUITests"
|
TARGET="HoneyDueUITests"
|
||||||
DESTINATION="platform=iOS Simulator,name=iPhone 17 Pro"
|
DESTINATION="platform=iOS Simulator,name=iPhone 17 Pro"
|
||||||
WORKERS=8
|
# Data isolation removed cross-suite races, but each test now relaunches the
|
||||||
|
# app + creates an account, so the bottleneck is CPU/simulator. 4 is the reliable
|
||||||
|
# sweet spot on a Mac mini (8 thrashed the simulators -> UI timeouts). Override via arg 2.
|
||||||
|
WORKERS=4
|
||||||
|
|
||||||
# Suites that run in their own phases — excluded from the parallel phase.
|
# Suites that run in their own phases — excluded from the parallel phase.
|
||||||
PHASE_MANAGED=(
|
PHASE_MANAGED=(
|
||||||
|
|||||||
Reference in New Issue
Block a user