admin/honeyDueKMP
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
00a217e8c8d7eafb8eba5f76c0130ef638422dfb
honeyDueKMP/03_14_26_uiresults.md
Trey T 00e9ed0a96 Add localization strings and iOS test infrastructure 
- Expand Localizable.xcstrings with 426 new localization entries
- Add xctestplan files (CI, Cleanup, Parallel, Seed) for structured test runs
- Add run_ui_tests.sh script for UI test execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-01 20:32:09 -05:00

16 KiB
Raw Blame History

HoneyDue UI Test Results - March 14, 2026

Branch: rename/honeydue Device: iPhone 16 Simulator (iOS 26.2) Parallel: No (sequential execution) Machine: Apple M1, 8GB RAM

Summary

Status Count
Passed 71
Failed 168
Skipped 15
Total 254

Pass Rate: 28.0%

Results by Suite

Suite Passed Failed Skipped Total Pass Rate
AccessibilityTests 6 0 0 6 100%
AppLaunchTests 2 0 0 2 100%
StabilityTests 6 0 0 6 100%
SimpleLoginTest 2 0 0 2 100%
Suite3_ResidenceRebuildTests 9 0 0 9 100%
Suite0_OnboardingRebuildTests 2 0 3 5 100%*
OnboardingTests 10 3 0 13 77%
Suite2_AuthenticationRebuildTests 4 2 0 6 67%
Suite2_AuthenticationTests 4 2 0 6 67%
AuthCriticalPathTests 3 2 0 5 60%
Suite1_RegistrationTests 6 5 0 11 55%
Suite9_IntegrationE2ETests 2 5 0 7 29%
SmokeTests 1 4 0 5 20%
Suite4_ComprehensiveResidenceTests 2 16 0 18 11%
Suite6_ComprehensiveTaskTests 2 13 0 15 13%
Suite10_ComprehensiveE2ETests 1 8 0 9 11%
Suite5_TaskTests 1 9 0 10 10%
Suite7_ContractorTests 2 18 0 20 10%
AuthenticationTests 10 6 0 16 63%
Suite1_RegistrationRebuildTests 0 2 12 14 0%*
ContractorIntegrationTests 0 5 0 5 0%
DataLayerTests 0 10 0 10 0%
DocumentIntegrationTests 0 4 0 4 0%
NavigationCriticalPathTests 0 10 0 10 0%
PasswordResetTests 0 5 0 5 0%
ResidenceIntegrationTests 0 5 0 5 0%
Suite0_OnboardingTests 0 1 0 1 0%
Suite3_ResidenceTests 0 6 0 6 0%
Suite8_DocumentWarrantyTests 0 25 0 25 0%
TaskIntegrationTests 0 5 0 5 0%

*Pass rate excludes skipped tests

Fully Passing Suites (6)

  • AccessibilityTests (6/6)
  • AppLaunchTests (2/2)
  • StabilityTests (6/6)
  • SimpleLoginTest (2/2)
  • Suite3_ResidenceRebuildTests (9/9)
  • Suite0_OnboardingRebuildTests (2/2 run, 3 skipped)

Fully Failing Suites (10)

  • ContractorIntegrationTests (0/5)
  • DataLayerTests (0/10)
  • DocumentIntegrationTests (0/4)
  • NavigationCriticalPathTests (0/10)
  • PasswordResetTests (0/5)
  • ResidenceIntegrationTests (0/5)
  • Suite0_OnboardingTests (0/1)
  • Suite3_ResidenceTests (0/6)
  • Suite8_DocumentWarrantyTests (0/25)
  • TaskIntegrationTests (0/5)

Detailed Results

AccessibilityTests (6 passed, 0 failed)

Test Result Time
testA001_OnboardingPrimaryControlsAreReachable PASSED 16.8s
testA002_LoginControlsRemainOperable PASSED 26.9s
testA003_CoreControlsExposeIdentifiers PASSED 15.8s
testA004_ValuePropsScreenControlsAreReachable PASSED 14.8s
testA005_NameResidenceScreenControlsAreReachable PASSED 20.0s
testA006_CreateAccountScreenControlsAreReachable PASSED 23.0s

AppLaunchTests (2 passed, 0 failed)

Test Result Time
testF001_ColdLaunchShowsOnboardingWelcome PASSED 12.3s
testF002_ColdLaunchShowsPrimaryOnboardingActions PASSED 9.7s

AuthCriticalPathTests (3 passed, 2 failed)

Test Result Time
testForgotPasswordButtonExists PASSED 21.6s
testLoginWithInvalidCredentials PASSED 21.0s
testLoginWithValidCredentials FAILED 31.4s
testLogoutFlow FAILED 36.0s
testSignUpButtonNavigatesToRegistration PASSED 21.0s

AuthenticationTests (10 passed, 6 failed)

Test Result Time
test08_invalidatedTokenRedirectsToLogin FAILED 54.5s
testF201_OnboardingLoginEntryShowsLoginScreen PASSED 15.8s
testF202_LoginScreenCanTogglePasswordVisibility PASSED 19.1s
testF203_RegisterSheetCanOpenAndDismiss FAILED 23.4s
testF204_RegisterFormAcceptsInput FAILED 23.7s
testF205_LoginButtonDisabledWhenCredentialsAreEmpty PASSED 16.1s
testF206_ForgotPasswordButtonIsAccessible PASSED 16.4s
testF207_LoginScreenShowsAllExpectedElements PASSED 15.2s
testF208_RegisterFormShowsAllRequiredFields FAILED 24.4s
testF209_ForgotPasswordNavigatesToResetFlow PASSED 18.2s

ContractorIntegrationTests (0 passed, 5 failed)

Test Result Time
test20_toggleContractorFavorite FAILED 0.4s
test21_contractorByResidenceFilter FAILED 0.3s
testCON002_CreateContractorMinimalFields FAILED 0.2s
testCON005_EditContractor FAILED 0.2s
testCON006_DeleteContractor FAILED 0.2s

DataLayerTests (0 passed, 10 failed)

Test Result Time
test08_diskPersistencePreservesLookupsAfterRestart FAILED 0.2s
test09_themePersistsAcrossRestart FAILED 0.2s
test10_completionHistoryLoadsAndIsSorted FAILED 0.2s
testDATA001_LookupsInitializeAfterLogin FAILED 0.2s
testDATA002_ETagRefreshHandles304 FAILED 0.2s
testDATA003_LegacyFallbackStillLoadsCoreLookups FAILED 0.2s
testDATA004_CacheTimeoutAndForceRefresh FAILED 0.2s
testDATA005_LogoutClearsUserDataButRetainsTheme FAILED 0.2s
testDATA006_LookupsPersistAfterAppRestart FAILED 0.2s
testDATA007_LookupMapListConsistency FAILED 0.2s

DocumentIntegrationTests (0 passed, 4 failed)

Test Result Time
test22_documentImageSectionExists FAILED 0.2s
testDOC002_CreateDocumentWithRequiredFields FAILED 0.2s
testDOC004_EditDocument FAILED 0.2s
testDOC005_DeleteDocument FAILED 0.2s

NavigationCriticalPathTests (0 passed, 10 failed)

Test Result Time
testAllTabsExist FAILED 45.9s
testContractorAddButtonExists FAILED 45.9s
testDocumentAddButtonExists FAILED 45.9s
testNavigateBackToResidencesTab FAILED 45.8s
testNavigateToContractorsTab FAILED 46.2s
testNavigateToDocumentsTab FAILED 45.6s
testNavigateToTasksTab FAILED 45.8s
testResidenceAddButtonExists FAILED 45.8s
testSettingsButtonExists FAILED 45.5s
testTaskAddButtonExists FAILED 45.8s

OnboardingTests (10 passed, 3 failed)

Test Result Time
testF101_StartFreshFlowReachesCreateAccount PASSED 20.2s
testF102_JoinExistingFlowGoesToCreateAccount PASSED 11.6s
testF103_BackNavigationFromNameResidenceReturnsToValueProps PASSED 16.2s
testF104_SkipOnValuePropsMovesToNameResidence PASSED 13.9s
testF105_JoinExistingFlowSkipsValuePropsAndNameResidence PASSED 11.2s
testF106_NameResidenceFieldAcceptsInput PASSED 15.7s
testF107_ProgressIndicatorVisibleDuringOnboarding PASSED 12.8s
testF108_BackFromCreateAccountNavigatesToPreviousStep PASSED 22.5s
testF110_startFreshCreatesResidenceAfterVerification FAILED 29.1s
testF111_completedOnboardingBypassedOnRelaunch FAILED 52.9s

PasswordResetTests (0 passed, 5 failed)

Test Result Time
test03_verifyResetCodeSuccess FAILED 35.4s
test04_resetPasswordSuccessAndLogin FAILED 35.4s
testAUTH015_VerifyResetCodeSuccessPath FAILED 35.3s
testAUTH016_ResetPasswordSuccess FAILED 36.1s
testAUTH017_MismatchedPasswordBlocked FAILED 35.3s

ResidenceIntegrationTests (0 passed, 5 failed)

Test Result Time
test18_setPrimaryResidence FAILED 0.4s
test19_doubleSubmitProtection FAILED 0.3s
testRES_CreateResidenceAppearsInList FAILED 0.2s
testRES_DeleteResidenceRemovesFromList FAILED 0.2s
testRES_EditResidenceUpdatesInList FAILED 0.2s

SimpleLoginTest (2 passed, 0 failed)

Test Result Time
testAppLaunchesAndShowsLoginScreen PASSED 24.3s
testCanTypeInLoginFields PASSED 27.9s

SmokeTests (1 passed, 4 failed)

Test Result Time
testAppLaunches FAILED 25.8s
testLoginScreenElements PASSED 20.7s
testLoginWithExistingCredentials FAILED 30.7s
testMainTabsExistAfterLogin FAILED 35.6s
testTabNavigation FAILED 35.4s

StabilityTests (6 passed, 0 failed)

Test Result Time
testP001_RapidOnboardingNavigationDoesNotCrash PASSED 31.6s
testP002_RepeatedForwardNavigationRemainsResponsive PASSED 53.1s
testP003_RapidDoubleTapOnValuePropsContinueLandsOnNameResidence PASSED 13.7s
testP004_StartFreshThenBackToWelcomeThenJoinExistingDoesNotCorruptState PASSED 17.0s
testP005_RepeatedLoginNavigationRemainsStable PASSED 41.4s
testP010_retryButtonExistsOnErrorState PASSED 21.1s

Suite0_OnboardingRebuildTests (2 passed, 0 failed, 3 skipped)

Test Result Time
testR001_onboardingWelcomeLoadsAndCanNavigateToLoginEntry PASSED 12.5s
testR002_startFreshFlowReachesCreateAccount PASSED 19.0s
testR003_createAccountExpandedFormFieldsAreInteractable SKIPPED 6.5s
testR004_emailFieldCanFocusAndAcceptTyping SKIPPED 6.2s
testR005_createAccountContinueOnlyAfterValidInputs SKIPPED 6.3s

Suite0_OnboardingTests (0 passed, 1 failed)

Test Result Time
test_onboarding FAILED 33.1s

Suite1_RegistrationRebuildTests (0 passed, 2 failed, 12 skipped)

Test Result Time
testR101_registerFormCanOpenFromLogin FAILED 29.9s
testR102_registerFormAcceptsValidInput FAILED 29.1s
testR103-R114 (12 tests) SKIPPED ~5s each

Suite1_RegistrationTests (6 passed, 5 failed)

Test Result Time
test01_registrationScreenElements PASSED 40.8s
test02_cancelRegistration PASSED 46.7s
test03_registrationWithEmptyFields PASSED 45.0s
test04_registrationWithInvalidEmail PASSED 54.8s
test05_registrationWithMismatchedPasswords PASSED 55.9s
test06_registrationWithWeakPassword PASSED 55.4s
test07_successfulRegistrationAndVerification FAILED 64.5s
test09_registrationWithInvalidVerificationCode FAILED 95.4s
test10_verificationCodeFieldValidation FAILED 95.6s
test11_appRelaunchWithUnverifiedUser FAILED 96.5s
test12_logoutFromVerificationScreen FAILED 96.6s

Suite2_AuthenticationRebuildTests (4 passed, 2 failed)

Test Result Time
testR201_loginScreenLoadsFromOnboardingEntry PASSED 29.9s
testR202_validCredentialsSubmitFromLogin PASSED 33.4s
testR203_validLoginTransitionsToMainAppRoot PASSED 38.9s
testR204_mainAppHasExpectedPrimaryTabsAfterLogin PASSED 24.2s
testR205_logoutFromMainAppReturnsToLoginRoot FAILED 46.4s
testR206_postLogoutMainAppIsNoLongerAccessible FAILED 43.8s

Suite2_AuthenticationTests (4 passed, 2 failed)

Test Result Time
test01_loginWithInvalidCredentials PASSED 22.8s
test02_loginWithValidCredentials FAILED 30.8s
test03_passwordVisibilityToggle PASSED 15.6s
test04_navigationToSignUp PASSED 14.1s
test05_forgotPasswordNavigation PASSED 14.0s
test06_logout FAILED 29.7s

Suite3_ResidenceRebuildTests (9 passed, 0 failed)

Test Result Time
testR301_authenticatedPreconditionCanReachMainApp PASSED 24.2s
testR302_residencesTabIsPresentAndNavigable PASSED 21.9s
testR303_residencesListLoadsAfterTabSelection PASSED 22.8s
testR304_openAddResidenceFormFromResidencesList PASSED 25.1s
testR305_cancelAddResidenceReturnsToResidenceList PASSED 26.9s
testR306_createResidenceMinimalDataSubmitsSuccessfully PASSED 31.3s
testR307_newResidenceAppearsInResidenceList PASSED 31.1s
testR308_openResidenceDetailsFromResidenceList PASSED 32.6s
testR309_navigationAcrossPrimaryTabsAndBackToResidences PASSED 25.5s

Suite3_ResidenceTests (0 passed, 6 failed)

Test Result Time
test01_viewResidencesList FAILED 31.1s
test02_navigateToAddResidence FAILED 31.0s
test03_navigationBetweenTabs FAILED 31.5s
test04_cancelResidenceCreation FAILED 31.9s
test05_createResidenceWithMinimalData FAILED 31.1s
test06_viewResidenceDetails FAILED 31.4s

Suite4_ComprehensiveResidenceTests (2 passed, 16 failed)

Test Result Time
test01-16 (16 tests) FAILED ~36s each
test17_residenceListPerformance PASSED 78.1s
test18_residenceCreationPerformance PASSED 47.5s

Suite5_TaskTests (1 passed, 9 failed)

Test Result Time
test01_cancelTaskCreation FAILED 50.8s
test02_tasksTabExists FAILED 39.3s
test03_viewTasksList FAILED 53.2s
test04_addTaskButtonExists FAILED 50.4s
test05_navigateToAddTask FAILED 50.5s
test06_createBasicTask FAILED 50.7s
test07_viewTaskDetails PASSED 52.6s
test08_navigateToContractors FAILED 50.7s
test09_navigateToDocuments FAILED 51.0s
test10_navigateBetweenTabs FAILED 45.7s

Suite6_ComprehensiveTaskTests (2 passed, 13 failed)

Test Result Time
test01-13 (13 tests) FAILED ~43-56s each
test14_taskListPerformance PASSED 82.7s
test15_taskCreationPerformance PASSED 52.7s

Suite7_ContractorTests (2 passed, 18 failed)

Test Result Time
test01-18 (18 tests) FAILED ~36s each
test19_contractorListPerformance PASSED 78.6s
test20_contractorCreationPerformance PASSED 48.3s

Suite8_DocumentWarrantyTests (0 passed, 25 failed)

Test Result Time
test01-25 (all 25 tests) FAILED ~46-50s each

Suite9_IntegrationE2ETests (2 passed, 5 failed)

Test Result Time
test01_authenticationFlow FAILED 42.8s
test02_residenceCRUDFlow FAILED 40.9s
test03_taskLifecycleFlow FAILED 57.5s
test04_kanbanColumnDistribution FAILED 38.0s
test05_crossUserAccessControl FAILED 37.7s
test06_lookupDataAvailable PASSED 41.8s
test07_residenceSharingUIElements PASSED 42.0s

TaskIntegrationTests (0 passed, 5 failed)

Test Result Time
test15_uncancelRestorescancelledTask FAILED 0.4s
test16_createTaskFromTemplate FAILED 0.2s
testTASK_CreateTaskAppearsInList FAILED 0.2s
testTASK010_UncancelTaskFlow FAILED 0.2s
testTASK012_DeleteTaskUpdatesViews FAILED 0.2s

Observations

Patterns in Failures

  1. Integration tests fail instantly (~0.2s): ContractorIntegrationTests, DataLayerTests, DocumentIntegrationTests, ResidenceIntegrationTests, TaskIntegrationTests all fail in < 0.5s, suggesting they crash on setup or have missing preconditions.

  2. NavigationCriticalPathTests all timeout at ~45s: These require authenticated login but consistently fail at the same timeout, likely unable to complete login flow.

  3. Suite3-8 (authenticated CRUD tests) fail at ~36-50s: All authenticated tests that use the older test patterns fail with similar timeouts, suggesting the login/auth flow in these older suites is broken.

  4. Rebuild suites pass where old suites fail: Suite3_ResidenceRebuildTests passes 9/9 while Suite3_ResidenceTests fails 6/6. The rebuild suites use the updated AuthenticatedTestCase framework.

  5. Pre-auth tests pass reliably: Onboarding, accessibility, stability, app launch, and simple login tests all pass because they don't require authentication.

  6. Logout tests consistently fail: Both testR205_logoutFromMainAppReturnsToLoginRoot and test06_logout fail across multiple suites.

Build Fix Applied

Fixed TEST_HOST casing mismatch in project.pbxproj: changed HoneyDue.app/HoneyDue to honeyDue.app/honeyDue to match the renamed product.

Fixed SubscriptionGatingTests compile error: added missing tier, isActive, trialStart, trialEnd, trialActive, subscriptionSource parameters to SubscriptionStatus constructor.

Reference in New Issue View Git Blame Copy Permalink