Fix per-test isolation flakiness: relaunch instead of UI logout; 4 workers

The first full 8-worker run surfaced 52 failures, 28 of them "Failed to log out"
(UITestHelpers:86) — forcing a profile-navigation logout between every test (each
test = new account) is fragile, and 8 parallel simulator clones thrashed the
machine (the remaining failures were UI timeouts under that load).

- AuthenticatedUITestCase: relaunchBetweenTests = true. A fresh app launch with
  --reset-state lands on the login screen, so each test logs in as its own account
  with NO UI logout between tests. Removed the ensureLoggedOut call.
- run_ui_tests.sh: default workers 8 -> 4 (reliable on a Mac mini; each test now
  relaunches + creates an account, so the bottleneck is CPU/simulator).

Verified: ContractorUITests (was ~15 logout failures) now passes at 4 workers,
0 leaked accounts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Trey T
2026-06-05 17:42:38 -05:00
parent 7cdd88b11a
commit 091248f30f
2 changed files with 14 additions and 6 deletions
@@ -17,6 +17,12 @@ class AuthenticatedUITestCase: BaseUITestCase {
/// (sets OnboardingState.hasCompletedOnboarding) so login lands on main tabs.
override var completeOnboarding: Bool { true }
/// Per-test isolation relaunches the app fresh for every test. With
/// --reset-state this lands on the login screen, so each test logs in as its
/// own fresh account WITHOUT a fragile UI logout between tests (the old
/// logout-via-profile path was the #1 source of flakes under load).
override var relaunchBetweenTests: Bool { true }
/// Credentials for the Kratos APP identity used to seed data over the API.
///
/// TWO DIFFERENT "admin@honeydue.com" EXIST do NOT "fix" Test1234 to password123:
@@ -116,11 +122,10 @@ class AuthenticatedUITestCase: BaseUITestCase {
try super.setUpWithError()
if usesFreshAccount {
// Per-test isolation: every test logs in as its OWN fresh, pre-verified
// account, seeds under its token, and deletes it in teardown. The app
// may be reused from a previous test (still logged in as that test's
// account), so always log out first.
UITestHelpers.ensureLoggedOut(app: app)
// Per-test isolation: relaunchBetweenTests gives every test a fresh
// app launch that lands on the login screen (--reset-state), so we
// log in as a brand-new pre-verified account, seed under its token,
// and delete it in teardown. No UI logout between tests.
let acct = TestAccount.create(domain: accountDomain)
account = acct
session = acct.session