admin/honeyDueKMP
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
Files
c57743dca0deed363728e8ff4edef8046fc3438c
honeyDueKMP/03_14_26_uiresults.md
Trey T 00e9ed0a96 Add localization strings and iOS test infrastructure 
- Expand Localizable.xcstrings with 426 new localization entries
- Add xctestplan files (CI, Cleanup, Parallel, Seed) for structured test runs
- Add run_ui_tests.sh script for UI test execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-01 20:32:09 -05:00

418 lines
16 KiB
Markdown
Raw Blame History

# HoneyDue UI Test Results - March 14, 2026
**Branch:** `rename/honeydue`
**Device:** iPhone 16 Simulator (iOS 26.2)
**Parallel:** No (sequential execution)
**Machine:** Apple M1, 8GB RAM
## Summary
| Status | Count |
|--------|-------|
| Passed | 71 |
| Failed | 168 |
| Skipped | 15 |
| **Total** | **254** |
**Pass Rate: 28.0%**
---
## Results by Suite
| Suite | Passed | Failed | Skipped | Total | Pass Rate |
|-------|--------|--------|---------|-------|-----------|
| AccessibilityTests | 6 | 0 | 0 | 6 | 100% |
| AppLaunchTests | 2 | 0 | 0 | 2 | 100% |
| StabilityTests | 6 | 0 | 0 | 6 | 100% |
| SimpleLoginTest | 2 | 0 | 0 | 2 | 100% |
| Suite3_ResidenceRebuildTests | 9 | 0 | 0 | 9 | 100% |
| Suite0_OnboardingRebuildTests | 2 | 0 | 3 | 5 | 100%* |
| OnboardingTests | 10 | 3 | 0 | 13 | 77% |
| Suite2_AuthenticationRebuildTests | 4 | 2 | 0 | 6 | 67% |
| Suite2_AuthenticationTests | 4 | 2 | 0 | 6 | 67% |
| AuthCriticalPathTests | 3 | 2 | 0 | 5 | 60% |
| Suite1_RegistrationTests | 6 | 5 | 0 | 11 | 55% |
| Suite9_IntegrationE2ETests | 2 | 5 | 0 | 7 | 29% |
| SmokeTests | 1 | 4 | 0 | 5 | 20% |
| Suite4_ComprehensiveResidenceTests | 2 | 16 | 0 | 18 | 11% |
| Suite6_ComprehensiveTaskTests | 2 | 13 | 0 | 15 | 13% |
| Suite10_ComprehensiveE2ETests | 1 | 8 | 0 | 9 | 11% |
| Suite5_TaskTests | 1 | 9 | 0 | 10 | 10% |
| Suite7_ContractorTests | 2 | 18 | 0 | 20 | 10% |
| AuthenticationTests | 10 | 6 | 0 | 16 | 63% |
| Suite1_RegistrationRebuildTests | 0 | 2 | 12 | 14 | 0%* |
| ContractorIntegrationTests | 0 | 5 | 0 | 5 | 0% |
| DataLayerTests | 0 | 10 | 0 | 10 | 0% |
| DocumentIntegrationTests | 0 | 4 | 0 | 4 | 0% |
| NavigationCriticalPathTests | 0 | 10 | 0 | 10 | 0% |
| PasswordResetTests | 0 | 5 | 0 | 5 | 0% |
| ResidenceIntegrationTests | 0 | 5 | 0 | 5 | 0% |
| Suite0_OnboardingTests | 0 | 1 | 0 | 1 | 0% |
| Suite3_ResidenceTests | 0 | 6 | 0 | 6 | 0% |
| Suite8_DocumentWarrantyTests | 0 | 25 | 0 | 25 | 0% |
| TaskIntegrationTests | 0 | 5 | 0 | 5 | 0% |
*Pass rate excludes skipped tests
---
## Fully Passing Suites (6)
- **AccessibilityTests** (6/6)
- **AppLaunchTests** (2/2)
- **StabilityTests** (6/6)
- **SimpleLoginTest** (2/2)
- **Suite3_ResidenceRebuildTests** (9/9)
- **Suite0_OnboardingRebuildTests** (2/2 run, 3 skipped)
## Fully Failing Suites (10)
- **ContractorIntegrationTests** (0/5)
- **DataLayerTests** (0/10)
- **DocumentIntegrationTests** (0/4)
- **NavigationCriticalPathTests** (0/10)
- **PasswordResetTests** (0/5)
- **ResidenceIntegrationTests** (0/5)
- **Suite0_OnboardingTests** (0/1)
- **Suite3_ResidenceTests** (0/6)
- **Suite8_DocumentWarrantyTests** (0/25)
- **TaskIntegrationTests** (0/5)
---
## Detailed Results
### AccessibilityTests (6 passed, 0 failed)
| Test | Result | Time |
|------|--------|------|
| testA001_OnboardingPrimaryControlsAreReachable | PASSED | 16.8s |
| testA002_LoginControlsRemainOperable | PASSED | 26.9s |
| testA003_CoreControlsExposeIdentifiers | PASSED | 15.8s |
| testA004_ValuePropsScreenControlsAreReachable | PASSED | 14.8s |
| testA005_NameResidenceScreenControlsAreReachable | PASSED | 20.0s |
| testA006_CreateAccountScreenControlsAreReachable | PASSED | 23.0s |
### AppLaunchTests (2 passed, 0 failed)
| Test | Result | Time |
|------|--------|------|
| testF001_ColdLaunchShowsOnboardingWelcome | PASSED | 12.3s |
| testF002_ColdLaunchShowsPrimaryOnboardingActions | PASSED | 9.7s |
### AuthCriticalPathTests (3 passed, 2 failed)
| Test | Result | Time |
|------|--------|------|
| testForgotPasswordButtonExists | PASSED | 21.6s |
| testLoginWithInvalidCredentials | PASSED | 21.0s |
| testLoginWithValidCredentials | FAILED | 31.4s |
| testLogoutFlow | FAILED | 36.0s |
| testSignUpButtonNavigatesToRegistration | PASSED | 21.0s |
### AuthenticationTests (10 passed, 6 failed)
| Test | Result | Time |
|------|--------|------|
| test08_invalidatedTokenRedirectsToLogin | FAILED | 54.5s |
| testF201_OnboardingLoginEntryShowsLoginScreen | PASSED | 15.8s |
| testF202_LoginScreenCanTogglePasswordVisibility | PASSED | 19.1s |
| testF203_RegisterSheetCanOpenAndDismiss | FAILED | 23.4s |
| testF204_RegisterFormAcceptsInput | FAILED | 23.7s |
| testF205_LoginButtonDisabledWhenCredentialsAreEmpty | PASSED | 16.1s |
| testF206_ForgotPasswordButtonIsAccessible | PASSED | 16.4s |
| testF207_LoginScreenShowsAllExpectedElements | PASSED | 15.2s |
| testF208_RegisterFormShowsAllRequiredFields | FAILED | 24.4s |
| testF209_ForgotPasswordNavigatesToResetFlow | PASSED | 18.2s |
### ContractorIntegrationTests (0 passed, 5 failed)
| Test | Result | Time |
|------|--------|------|
| test20_toggleContractorFavorite | FAILED | 0.4s |
| test21_contractorByResidenceFilter | FAILED | 0.3s |
| testCON002_CreateContractorMinimalFields | FAILED | 0.2s |
| testCON005_EditContractor | FAILED | 0.2s |
| testCON006_DeleteContractor | FAILED | 0.2s |
### DataLayerTests (0 passed, 10 failed)
| Test | Result | Time |
|------|--------|------|
| test08_diskPersistencePreservesLookupsAfterRestart | FAILED | 0.2s |
| test09_themePersistsAcrossRestart | FAILED | 0.2s |
| test10_completionHistoryLoadsAndIsSorted | FAILED | 0.2s |
| testDATA001_LookupsInitializeAfterLogin | FAILED | 0.2s |
| testDATA002_ETagRefreshHandles304 | FAILED | 0.2s |
| testDATA003_LegacyFallbackStillLoadsCoreLookups | FAILED | 0.2s |
| testDATA004_CacheTimeoutAndForceRefresh | FAILED | 0.2s |
| testDATA005_LogoutClearsUserDataButRetainsTheme | FAILED | 0.2s |
| testDATA006_LookupsPersistAfterAppRestart | FAILED | 0.2s |
| testDATA007_LookupMapListConsistency | FAILED | 0.2s |
### DocumentIntegrationTests (0 passed, 4 failed)
| Test | Result | Time |
|------|--------|------|
| test22_documentImageSectionExists | FAILED | 0.2s |
| testDOC002_CreateDocumentWithRequiredFields | FAILED | 0.2s |
| testDOC004_EditDocument | FAILED | 0.2s |
| testDOC005_DeleteDocument | FAILED | 0.2s |
### NavigationCriticalPathTests (0 passed, 10 failed)
| Test | Result | Time |
|------|--------|------|
| testAllTabsExist | FAILED | 45.9s |
| testContractorAddButtonExists | FAILED | 45.9s |
| testDocumentAddButtonExists | FAILED | 45.9s |
| testNavigateBackToResidencesTab | FAILED | 45.8s |
| testNavigateToContractorsTab | FAILED | 46.2s |
| testNavigateToDocumentsTab | FAILED | 45.6s |
| testNavigateToTasksTab | FAILED | 45.8s |
| testResidenceAddButtonExists | FAILED | 45.8s |
| testSettingsButtonExists | FAILED | 45.5s |
| testTaskAddButtonExists | FAILED | 45.8s |
### OnboardingTests (10 passed, 3 failed)
| Test | Result | Time |
|------|--------|------|
| testF101_StartFreshFlowReachesCreateAccount | PASSED | 20.2s |
| testF102_JoinExistingFlowGoesToCreateAccount | PASSED | 11.6s |
| testF103_BackNavigationFromNameResidenceReturnsToValueProps | PASSED | 16.2s |
| testF104_SkipOnValuePropsMovesToNameResidence | PASSED | 13.9s |
| testF105_JoinExistingFlowSkipsValuePropsAndNameResidence | PASSED | 11.2s |
| testF106_NameResidenceFieldAcceptsInput | PASSED | 15.7s |
| testF107_ProgressIndicatorVisibleDuringOnboarding | PASSED | 12.8s |
| testF108_BackFromCreateAccountNavigatesToPreviousStep | PASSED | 22.5s |
| testF110_startFreshCreatesResidenceAfterVerification | FAILED | 29.1s |
| testF111_completedOnboardingBypassedOnRelaunch | FAILED | 52.9s |
### PasswordResetTests (0 passed, 5 failed)
| Test | Result | Time |
|------|--------|------|
| test03_verifyResetCodeSuccess | FAILED | 35.4s |
| test04_resetPasswordSuccessAndLogin | FAILED | 35.4s |
| testAUTH015_VerifyResetCodeSuccessPath | FAILED | 35.3s |
| testAUTH016_ResetPasswordSuccess | FAILED | 36.1s |
| testAUTH017_MismatchedPasswordBlocked | FAILED | 35.3s |
### ResidenceIntegrationTests (0 passed, 5 failed)
| Test | Result | Time |
|------|--------|------|
| test18_setPrimaryResidence | FAILED | 0.4s |
| test19_doubleSubmitProtection | FAILED | 0.3s |
| testRES_CreateResidenceAppearsInList | FAILED | 0.2s |
| testRES_DeleteResidenceRemovesFromList | FAILED | 0.2s |
| testRES_EditResidenceUpdatesInList | FAILED | 0.2s |
### SimpleLoginTest (2 passed, 0 failed)
| Test | Result | Time |
|------|--------|------|
| testAppLaunchesAndShowsLoginScreen | PASSED | 24.3s |
| testCanTypeInLoginFields | PASSED | 27.9s |
### SmokeTests (1 passed, 4 failed)
| Test | Result | Time |
|------|--------|------|
| testAppLaunches | FAILED | 25.8s |
| testLoginScreenElements | PASSED | 20.7s |
| testLoginWithExistingCredentials | FAILED | 30.7s |
| testMainTabsExistAfterLogin | FAILED | 35.6s |
| testTabNavigation | FAILED | 35.4s |
### StabilityTests (6 passed, 0 failed)
| Test | Result | Time |
|------|--------|------|
| testP001_RapidOnboardingNavigationDoesNotCrash | PASSED | 31.6s |
| testP002_RepeatedForwardNavigationRemainsResponsive | PASSED | 53.1s |
| testP003_RapidDoubleTapOnValuePropsContinueLandsOnNameResidence | PASSED | 13.7s |
| testP004_StartFreshThenBackToWelcomeThenJoinExistingDoesNotCorruptState | PASSED | 17.0s |
| testP005_RepeatedLoginNavigationRemainsStable | PASSED | 41.4s |
| testP010_retryButtonExistsOnErrorState | PASSED | 21.1s |
### Suite0_OnboardingRebuildTests (2 passed, 0 failed, 3 skipped)
| Test | Result | Time |
|------|--------|------|
| testR001_onboardingWelcomeLoadsAndCanNavigateToLoginEntry | PASSED | 12.5s |
| testR002_startFreshFlowReachesCreateAccount | PASSED | 19.0s |
| testR003_createAccountExpandedFormFieldsAreInteractable | SKIPPED | 6.5s |
| testR004_emailFieldCanFocusAndAcceptTyping | SKIPPED | 6.2s |
| testR005_createAccountContinueOnlyAfterValidInputs | SKIPPED | 6.3s |
### Suite0_OnboardingTests (0 passed, 1 failed)
| Test | Result | Time |
|------|--------|------|
| test_onboarding | FAILED | 33.1s |
### Suite1_RegistrationRebuildTests (0 passed, 2 failed, 12 skipped)
| Test | Result | Time |
|------|--------|------|
| testR101_registerFormCanOpenFromLogin | FAILED | 29.9s |
| testR102_registerFormAcceptsValidInput | FAILED | 29.1s |
| testR103-R114 (12 tests) | SKIPPED | ~5s each |
### Suite1_RegistrationTests (6 passed, 5 failed)
| Test | Result | Time |
|------|--------|------|
| test01_registrationScreenElements | PASSED | 40.8s |
| test02_cancelRegistration | PASSED | 46.7s |
| test03_registrationWithEmptyFields | PASSED | 45.0s |
| test04_registrationWithInvalidEmail | PASSED | 54.8s |
| test05_registrationWithMismatchedPasswords | PASSED | 55.9s |
| test06_registrationWithWeakPassword | PASSED | 55.4s |
| test07_successfulRegistrationAndVerification | FAILED | 64.5s |
| test09_registrationWithInvalidVerificationCode | FAILED | 95.4s |
| test10_verificationCodeFieldValidation | FAILED | 95.6s |
| test11_appRelaunchWithUnverifiedUser | FAILED | 96.5s |
| test12_logoutFromVerificationScreen | FAILED | 96.6s |
### Suite2_AuthenticationRebuildTests (4 passed, 2 failed)
| Test | Result | Time |
|------|--------|------|
| testR201_loginScreenLoadsFromOnboardingEntry | PASSED | 29.9s |
| testR202_validCredentialsSubmitFromLogin | PASSED | 33.4s |
| testR203_validLoginTransitionsToMainAppRoot | PASSED | 38.9s |
| testR204_mainAppHasExpectedPrimaryTabsAfterLogin | PASSED | 24.2s |
| testR205_logoutFromMainAppReturnsToLoginRoot | FAILED | 46.4s |
| testR206_postLogoutMainAppIsNoLongerAccessible | FAILED | 43.8s |
### Suite2_AuthenticationTests (4 passed, 2 failed)
| Test | Result | Time |
|------|--------|------|
| test01_loginWithInvalidCredentials | PASSED | 22.8s |
| test02_loginWithValidCredentials | FAILED | 30.8s |
| test03_passwordVisibilityToggle | PASSED | 15.6s |
| test04_navigationToSignUp | PASSED | 14.1s |
| test05_forgotPasswordNavigation | PASSED | 14.0s |
| test06_logout | FAILED | 29.7s |
### Suite3_ResidenceRebuildTests (9 passed, 0 failed)
| Test | Result | Time |
|------|--------|------|
| testR301_authenticatedPreconditionCanReachMainApp | PASSED | 24.2s |
| testR302_residencesTabIsPresentAndNavigable | PASSED | 21.9s |
| testR303_residencesListLoadsAfterTabSelection | PASSED | 22.8s |
| testR304_openAddResidenceFormFromResidencesList | PASSED | 25.1s |
| testR305_cancelAddResidenceReturnsToResidenceList | PASSED | 26.9s |
| testR306_createResidenceMinimalDataSubmitsSuccessfully | PASSED | 31.3s |
| testR307_newResidenceAppearsInResidenceList | PASSED | 31.1s |
| testR308_openResidenceDetailsFromResidenceList | PASSED | 32.6s |
| testR309_navigationAcrossPrimaryTabsAndBackToResidences | PASSED | 25.5s |
### Suite3_ResidenceTests (0 passed, 6 failed)
| Test | Result | Time |
|------|--------|------|
| test01_viewResidencesList | FAILED | 31.1s |
| test02_navigateToAddResidence | FAILED | 31.0s |
| test03_navigationBetweenTabs | FAILED | 31.5s |
| test04_cancelResidenceCreation | FAILED | 31.9s |
| test05_createResidenceWithMinimalData | FAILED | 31.1s |
| test06_viewResidenceDetails | FAILED | 31.4s |
### Suite4_ComprehensiveResidenceTests (2 passed, 16 failed)
| Test | Result | Time |
|------|--------|------|
| test01-16 (16 tests) | FAILED | ~36s each |
| test17_residenceListPerformance | PASSED | 78.1s |
| test18_residenceCreationPerformance | PASSED | 47.5s |
### Suite5_TaskTests (1 passed, 9 failed)
| Test | Result | Time |
|------|--------|------|
| test01_cancelTaskCreation | FAILED | 50.8s |
| test02_tasksTabExists | FAILED | 39.3s |
| test03_viewTasksList | FAILED | 53.2s |
| test04_addTaskButtonExists | FAILED | 50.4s |
| test05_navigateToAddTask | FAILED | 50.5s |
| test06_createBasicTask | FAILED | 50.7s |
| test07_viewTaskDetails | PASSED | 52.6s |
| test08_navigateToContractors | FAILED | 50.7s |
| test09_navigateToDocuments | FAILED | 51.0s |
| test10_navigateBetweenTabs | FAILED | 45.7s |
### Suite6_ComprehensiveTaskTests (2 passed, 13 failed)
| Test | Result | Time |
|------|--------|------|
| test01-13 (13 tests) | FAILED | ~43-56s each |
| test14_taskListPerformance | PASSED | 82.7s |
| test15_taskCreationPerformance | PASSED | 52.7s |
### Suite7_ContractorTests (2 passed, 18 failed)
| Test | Result | Time |
|------|--------|------|
| test01-18 (18 tests) | FAILED | ~36s each |
| test19_contractorListPerformance | PASSED | 78.6s |
| test20_contractorCreationPerformance | PASSED | 48.3s |
### Suite8_DocumentWarrantyTests (0 passed, 25 failed)
| Test | Result | Time |
|------|--------|------|
| test01-25 (all 25 tests) | FAILED | ~46-50s each |
### Suite9_IntegrationE2ETests (2 passed, 5 failed)
| Test | Result | Time |
|------|--------|------|
| test01_authenticationFlow | FAILED | 42.8s |
| test02_residenceCRUDFlow | FAILED | 40.9s |
| test03_taskLifecycleFlow | FAILED | 57.5s |
| test04_kanbanColumnDistribution | FAILED | 38.0s |
| test05_crossUserAccessControl | FAILED | 37.7s |
| test06_lookupDataAvailable | PASSED | 41.8s |
| test07_residenceSharingUIElements | PASSED | 42.0s |
### TaskIntegrationTests (0 passed, 5 failed)
| Test | Result | Time |
|------|--------|------|
| test15_uncancelRestorescancelledTask | FAILED | 0.4s |
| test16_createTaskFromTemplate | FAILED | 0.2s |
| testTASK_CreateTaskAppearsInList | FAILED | 0.2s |
| testTASK010_UncancelTaskFlow | FAILED | 0.2s |
| testTASK012_DeleteTaskUpdatesViews | FAILED | 0.2s |
---
## Observations
### Patterns in Failures
1. **Integration tests fail instantly (~0.2s):** ContractorIntegrationTests, DataLayerTests, DocumentIntegrationTests, ResidenceIntegrationTests, TaskIntegrationTests all fail in < 0.5s, suggesting they crash on setup or have missing preconditions.
2. **NavigationCriticalPathTests all timeout at ~45s:** These require authenticated login but consistently fail at the same timeout, likely unable to complete login flow.
3. **Suite3-8 (authenticated CRUD tests) fail at ~36-50s:** All authenticated tests that use the older test patterns fail with similar timeouts, suggesting the login/auth flow in these older suites is broken.
4. **Rebuild suites pass where old suites fail:** `Suite3_ResidenceRebuildTests` passes 9/9 while `Suite3_ResidenceTests` fails 6/6. The rebuild suites use the updated `AuthenticatedTestCase` framework.
5. **Pre-auth tests pass reliably:** Onboarding, accessibility, stability, app launch, and simple login tests all pass because they don't require authentication.
6. **Logout tests consistently fail:** Both `testR205_logoutFromMainAppReturnsToLoginRoot` and `test06_logout` fail across multiple suites.
### Build Fix Applied
Fixed `TEST_HOST` casing mismatch in `project.pbxproj`: changed `HoneyDue.app/HoneyDue` to `honeyDue.app/honeyDue` to match the renamed product.
Fixed `SubscriptionGatingTests` compile error: added missing `tier`, `isActive`, `trialStart`, `trialEnd`, `trialActive`, `subscriptionSource` parameters to `SubscriptionStatus` constructor.
Reference in New Issue View Git Blame Copy Permalink