Re-architect iOS XCUITest suite: per-test isolation + domain organization
Migrate the XCUITest suite off the legacy shared-account model (and the prior Django-style auth assumptions) to a parallel-safe, domain-organized architecture, validated end-to-end against the live Kratos stack. Isolation (parallel-safe by construction): - Core/Fixtures/TestAccount.swift: each test mints its own pre-verified Kratos identity (uit_<domain>_<uuid>@test.honeydue.local), logs in, seeds under its own token, and deletes the identity in teardown (cascading all data + clearing Kratos). No shared testuser; parallel workers no longer race. - AuthenticatedUITestCase rewritten to that model (member surface preserved); adds requiresResidence / seedAccountPreconditions to seed UI-gated data BEFORE login (a fresh account is empty at login). Organization (255 tests preserved, none dropped): - 21 domain suites under Auth/ Onboarding/ Residence/ Task/ Contractor/ Document/ Sharing/ Navigation/ Smoke/ CrossCutting/ E2E/, consistent <Domain>UITests naming. Removes the Suite1..11 / AAA_ / ZZ_ / Tests/Rebuild naming chaos and the overlapping task/residence/auth suites. Runner + test plans: - run_ui_tests.sh: Smoke gate -> Seed -> Parallel(8 workers) -> Sweep. The parallel phase runs the whole target minus phase-managed suites via -skip-testing, so new suites auto-include (no hand-maintained list to drift). Drops the 2-worker cap and Suite6 isolation (isolation made them moot). - HoneyDueUITests.xctestplan skips the 4 phase-managed suites; adds Smoke.xctestplan. Kratos auth fixes folded in (login/verify/reset endpoints removed under Kratos): real Mailpit verification codes replace the obsolete fixed "123456"; teardown deletes Kratos identities; admin-panel login uses the correct seeded password. Build green; isolation, parallelism, and the precondition/sharing migrations validated against the live stack (0 leaked accounts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Trey T
@@ -11,12 +11,64 @@ class AuthenticatedUITestCase: BaseUITestCase {
|
||||
|
||||
var needsAPISession: Bool { false }
|
||||
|
||||
/// Authenticated suites test the post-onboarding app. A freshly-seeded user
|
||||
/// has no residence, so without this the app routes to the onboarding flow
|
||||
/// after login instead of the main tabs. Launch with --complete-onboarding
|
||||
/// (sets OnboardingState.hasCompletedOnboarding) so login lands on main tabs.
|
||||
override var completeOnboarding: Bool { true }
|
||||
|
||||
/// Credentials for the Kratos APP identity used to seed data over the API.
|
||||
///
|
||||
/// ⚠️ TWO DIFFERENT "admin@honeydue.com" EXIST — do NOT "fix" Test1234 to password123:
|
||||
/// (a) Kratos APP identity — admin@honeydue.com / Test1234. Created by this class's
|
||||
/// `setUp` (and re-seeded by SuiteZZ). Used here for API data-seeding and login.
|
||||
/// (b) Admin-PANEL SQL super-admin — admin@honeydue.com / password123. A separate
|
||||
/// system, used ONLY by SuiteZZ_CleanupTests to call /admin/settings/clear-all-data.
|
||||
/// They happen to share an email but are unrelated. Changing Test1234 here would break
|
||||
/// all API seeding; changing password123 in SuiteZZ would break the data wipe.
|
||||
var apiCredentials: (username: String, password: String) {
|
||||
("admin", "Test1234")
|
||||
}
|
||||
|
||||
// MARK: - Account isolation
|
||||
|
||||
/// When `true` (default), each test mints its OWN unique, pre-verified
|
||||
/// Kratos account, logs in as it, seeds under its token, and deletes it in
|
||||
/// teardown — so suites are fully independent and parallel-safe. Override to
|
||||
/// `false` only in suites that must log in as a SPECIFIC seeded account
|
||||
/// (then also override `testCredentials`).
|
||||
var usesFreshAccount: Bool { true }
|
||||
|
||||
/// Short slug used in generated account emails (uit_<domain>_<uuid>@...),
|
||||
/// cosmetic for debugging. Defaults to the test class name.
|
||||
var accountDomain: String { String(describing: type(of: self)) }
|
||||
|
||||
/// The per-test isolated account (non-nil in fresh-account mode).
|
||||
private(set) var account: TestAccount?
|
||||
|
||||
/// Set `true` in suites whose UI gates on a residence existing (e.g. task
|
||||
/// or document creation). Seeds one residence BEFORE login so the app loads
|
||||
/// it on its post-login fetch; available to the test body as `seededResidence`.
|
||||
var requiresResidence: Bool { false }
|
||||
|
||||
/// The residence seeded as a precondition (when `requiresResidence`).
|
||||
private(set) var seededResidence: TestResidence?
|
||||
|
||||
/// Seed baseline data the UI gates on for this test's fresh account, BEFORE
|
||||
/// the app logs in (a fresh account is otherwise empty, so anything seeded
|
||||
/// after login is invisible until a manual refresh). Override to seed a full
|
||||
/// scenario (residence + tasks/documents); call `super` to keep the
|
||||
/// `requiresResidence` convenience.
|
||||
func seedAccountPreconditions(_ account: TestAccount) {
|
||||
if requiresResidence {
|
||||
seededResidence = account.seedResidence(name: "Precondition Home")
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - API Session
|
||||
|
||||
/// The authenticated session used for API seeding. In fresh-account mode
|
||||
/// this is the test's own account; in legacy mode it's `apiCredentials`.
|
||||
private(set) var session: TestSession!
|
||||
private(set) var cleaner: TestDataCleaner!
|
||||
|
||||
@@ -25,11 +77,16 @@ class AuthenticatedUITestCase: BaseUITestCase {
|
||||
override class func setUp() {
|
||||
super.setUp()
|
||||
guard TestAccountAPIClient.isBackendReachable() else { return }
|
||||
// Ensure both known test accounts exist (covers all subclass credential overrides)
|
||||
if TestAccountAPIClient.login(username: "testuser", password: "TestPass123!") == nil {
|
||||
// Ensure both known test accounts exist (covers all subclass credential overrides).
|
||||
// Kratos uses the EMAIL as the login identifier, so log in by email.
|
||||
// NOTE: the admin@honeydue.com / Test1234 created here is the Kratos APP identity
|
||||
// (system (a) in the `apiCredentials` doc above) — NOT the admin-panel SQL
|
||||
// super-admin (admin@honeydue.com / password123) that SuiteZZ uses for the data
|
||||
// wipe. Same email, separate systems; keep Test1234 here.
|
||||
if TestAccountAPIClient.login(username: "testuser@honeydue.com", password: "TestPass123!") == nil {
|
||||
_ = TestAccountAPIClient.createVerifiedAccount(username: "testuser", email: "testuser@honeydue.com", password: "TestPass123!")
|
||||
}
|
||||
if TestAccountAPIClient.login(username: "admin", password: "Test1234") == nil {
|
||||
if TestAccountAPIClient.login(username: "admin@honeydue.com", password: "Test1234") == nil {
|
||||
_ = TestAccountAPIClient.createVerifiedAccount(username: "admin", email: "admin@honeydue.com", password: "Test1234")
|
||||
}
|
||||
}
|
||||
@@ -58,31 +115,45 @@ class AuthenticatedUITestCase: BaseUITestCase {
|
||||
|
||||
try super.setUpWithError()
|
||||
|
||||
if usesFreshAccount {
|
||||
// Per-test isolation: every test logs in as its OWN fresh, pre-verified
|
||||
// account, seeds under its token, and deletes it in teardown. The app
|
||||
// may be reused from a previous test (still logged in as that test's
|
||||
// account), so always log out first.
|
||||
UITestHelpers.ensureLoggedOut(app: app)
|
||||
let acct = TestAccount.create(domain: accountDomain)
|
||||
account = acct
|
||||
session = acct.session
|
||||
cleaner = TestDataCleaner(token: acct.token)
|
||||
// Seed UI-gated baseline data BEFORE login so the app loads it on
|
||||
// its post-login fetch (a fresh account is otherwise empty).
|
||||
seedAccountPreconditions(acct)
|
||||
acct.login(into: app, timeout: loginTimeout)
|
||||
waitForMainApp()
|
||||
return
|
||||
}
|
||||
|
||||
// Legacy path: log in as a SPECIFIC seeded account (testCredentials),
|
||||
// optionally opening a separate API session (apiCredentials).
|
||||
let tabBar = app.tabBars.firstMatch
|
||||
let alreadyLoggedIn = tabBar.waitForExistence(timeout: defaultTimeout)
|
||||
|
||||
// Force-fresh path: log out (if needed) and re-authenticate per
|
||||
// test so every test starts with a freshly-issued JWT. Catches
|
||||
// server-side token invalidation that would otherwise surface
|
||||
// mid-suite as opaque 401s on the first mutation call.
|
||||
if forceFreshLoginPerTest {
|
||||
if alreadyLoggedIn {
|
||||
UITestHelpers.ensureLoggedOut(app: app)
|
||||
} else {
|
||||
UITestHelpers.ensureLoggedOut(app: app)
|
||||
}
|
||||
UITestHelpers.ensureLoggedOut(app: app)
|
||||
loginToMainApp()
|
||||
} else if !alreadyLoggedIn {
|
||||
// Legacy session-reuse path: only log in when not already in.
|
||||
UITestHelpers.ensureLoggedOut(app: app)
|
||||
loginToMainApp()
|
||||
}
|
||||
// (When `forceFreshLoginPerTest == false` AND we're already
|
||||
// logged in, fall through with the existing session.)
|
||||
|
||||
if needsAPISession {
|
||||
// Kratos uses the EMAIL as the login identifier. Subclasses still
|
||||
// declare seeded `apiCredentials` by short username (e.g. "admin"),
|
||||
// so normalize bare usernames to their "<username>@honeydue.com" email.
|
||||
let identifier = apiCredentials.username.contains("@")
|
||||
? apiCredentials.username
|
||||
: "\(apiCredentials.username)@honeydue.com"
|
||||
guard let apiSession = TestAccountManager.loginSeededAccount(
|
||||
username: apiCredentials.username,
|
||||
username: identifier,
|
||||
password: apiCredentials.password
|
||||
) else {
|
||||
XCTFail("Could not login API account '\(apiCredentials.username)'")
|
||||
@@ -94,7 +165,14 @@ class AuthenticatedUITestCase: BaseUITestCase {
|
||||
}
|
||||
|
||||
override func tearDownWithError() throws {
|
||||
cleaner?.cleanAll()
|
||||
// Deleting the per-test account cascades all of its data and clears the
|
||||
// Kratos identity in one call. In legacy mode there's no account, so
|
||||
// fall back to tracked-resource cleanup.
|
||||
if let account {
|
||||
account.delete()
|
||||
} else {
|
||||
cleaner?.cleanAll()
|
||||
}
|
||||
try super.tearDownWithError()
|
||||
}
|
||||
|
||||
@@ -107,7 +185,13 @@ class AuthenticatedUITestCase: BaseUITestCase {
|
||||
|
||||
let login = LoginScreenObject(app: app)
|
||||
login.waitForLoad(timeout: loginTimeout)
|
||||
login.enterUsername(creds.username)
|
||||
// Kratos uses the EMAIL as the login identifier. Subclasses still declare
|
||||
// testCredentials by short username (e.g. "admin"/"testuser"), so normalize
|
||||
// a bare username to "<username>@honeydue.com" for the app's login form.
|
||||
let identifier = creds.username.contains("@")
|
||||
? creds.username
|
||||
: "\(creds.username)@honeydue.com"
|
||||
login.enterUsername(identifier)
|
||||
login.enterPassword(creds.password)
|
||||
|
||||
let loginButton = app.buttons[AccessibilityIdentifiers.Authentication.loginButton]
|
||||
@@ -133,7 +217,24 @@ class AuthenticatedUITestCase: BaseUITestCase {
|
||||
RunLoop.current.run(until: Date().addingTimeInterval(0.5))
|
||||
}
|
||||
|
||||
XCTAssertTrue(tabBar.exists, "Expected tab bar after login with '\(testCredentials.username)'")
|
||||
if !tabBar.exists {
|
||||
XCTFail("Expected tab bar after login with '\(testCredentials.username)'. " +
|
||||
"Root state: " + Self.diagnoseRootState(app))
|
||||
}
|
||||
}
|
||||
|
||||
/// Diagnostic: report which RootView branch the app is parked on when
|
||||
/// the tab bar fails to appear after login. Helps distinguish a failed login
|
||||
/// (parked on ui.root.login) from a stuck verify-email gate.
|
||||
static func diagnoseRootState(_ app: XCUIApplication) -> String {
|
||||
let login = app.otherElements["ui.root.login"].exists
|
||||
let onboarding = app.otherElements["ui.root.onboarding"].exists
|
||||
let mainTabs = app.otherElements["ui.root.mainTabs"].exists
|
||||
let verifyCode = app.textFields[AccessibilityIdentifiers.Authentication.verificationCodeField].exists
|
||||
|| app.textFields[AccessibilityIdentifiers.Onboarding.verificationCodeField].exists
|
||||
let usernameField = app.textFields[AccessibilityIdentifiers.Authentication.usernameField].exists
|
||||
return "login=\(login) onboarding=\(onboarding) mainTabs=\(mainTabs) " +
|
||||
"verifyCodeField=\(verifyCode) usernameField=\(usernameField)"
|
||||
}
|
||||
|
||||
// MARK: - Tab Navigation
|
||||
|
||||
Reference in New Issue
Block a user