Phase 09.1: Fix Flaky Test When Ran In Parallel - 1 plan created - 2 total tasks defined - Ready for execution Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
6.6 KiB
phase, type
| phase | type |
|---|---|
| 09.1-fix-flaky-test-parallel | execute |
Purpose: Ensure test suite stability by resolving Swift Testing parallel execution state pollution. Output: All tests pass reliably in full suite runs, enabling consistent CI/CD testing.
<execution_context> ~/.claude/get-shit-done/workflows/execute-phase.md ./summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/codebase/TESTING.md @.planning/codebase/CONVENTIONS.mdPrior Phase Context
@.planning/phases/09-trip-planner-modes-tdd/09-01-SUMMARY.md @.planning/phases/09-trip-planner-modes-tdd/09-02-SUMMARY.md @.planning/phases/09-trip-planner-modes-tdd/09-03-SUMMARY.md
Affected Test Files
@SportsTimeTests/ScenarioAPlannerSwiftTests.swift @SportsTimeTests/ScenarioBPlannerTests.swift @SportsTimeTests/ScenarioCPlannerTests.swift
Problem Summary: 5 tests fail when run in full suite but pass individually:
plan_StopDepartureDate_IsLastGameDate()(ScenarioAPlannerSwiftTests)plan_ManyGames_HandledEfficiently()(ScenarioAPlannerSwiftTests)plan_ThreeSameDayGames_PicksFeasibleCombinations()(ScenarioAPlannerSwiftTests)plan_FillerSameDayAsAnchor_Excluded()(ScenarioBPlannerTests)plan_MustSeeGamesTooFarApart_Fails()(ScenarioBPlannerTests)
Root Cause: Swift Testing parallel execution + simulator state pollution. Tests share mutable state or depend on execution order.
Tech Stack Available:
- Swift Testing framework (iOS 26+)
.serializedtrait for sequential execution- Test isolation via setup/teardown
- Per-test state management
- Check for shared mutable state - Look for static properties, shared actors, or global state in test suites
- Check for execution order dependencies - Look for tests that depend on previous test side effects
- Check for async cleanup issues - Look for incomplete async operations or lingering tasks
Apply Swift Testing isolation patterns:
Option A: Add .serialized trait (preferred for quick fix):
@Suite("ScenarioAPlanner Tests", .serialized)
struct ScenarioAPlannerSwiftTests {
// Forces sequential execution, prevents parallel state pollution
}
Option B: Proper test isolation (if state pollution identified):
- Add per-test setup/teardown to reset shared state
- Use
init()anddeinitfor test suite lifecycle management - Ensure actors and services are created fresh per test
What to avoid and WHY:
- Don't just disable parallel testing globally - this slows down the entire suite
- Don't skip or comment out failing tests - they validate correct behavior
- Don't modify test expectations to make them pass - tests define correctness
Apply the minimal fix that ensures reliability. If .serialized fixes it, use that. If specific state needs cleanup, add proper isolation.
Run full test suite 3 times consecutively:
for i in 1 2 3; do
echo "Run $i:"
xcodebuild -project SportsTime.xcodeproj -scheme SportsTime -destination 'platform=iOS Simulator,name=iPhone 17,OS=26.2' test 2>&1 | grep -E "(Test Suite.*started|Test Suite.*passed|Test Suite.*failed|failed)"
done
All 5 previously flaky tests must pass in all 3 runs. All tests pass reliably in parallel execution, no flaky failures across 3 consecutive full suite runs
Task 2: Document root cause and solution in test files SportsTimeTests/ScenarioAPlannerSwiftTests.swift, SportsTimeTests/ScenarioBPlannerTests.swift, SportsTimeTests/ScenarioCPlannerTests.swift Add documentation comment to each modified test suite explaining: 1. Why the fix was needed (parallel execution state pollution) 2. What pattern was applied (`.serialized` trait or isolation strategy) 3. When it can be removed (if state pollution is ever eliminated)Example:
// MARK: - Swift Testing Isolation
// `.serialized` trait applied to prevent parallel execution state pollution.
// Tests pass individually but fail in parallel due to shared simulator state.
// Can be removed if test isolation is improved in future phases.
@Suite("ScenarioAPlanner Tests", .serialized)
struct ScenarioAPlannerSwiftTests {
This ensures future developers understand the tradeoff (sequential vs parallel) and the reasoning. Documentation comments present in all modified test files All modified test suites have clear comments explaining the isolation strategy and reasoning
Before declaring phase complete: - [ ] Full test suite runs successfully 3 times consecutively - [ ] All 5 previously flaky tests now pass reliably - [ ] No new test failures introduced - [ ] Documentation comments explain isolation strategy - [ ] Git commit created with proper format<success_criteria>
- All tasks completed
- All verification checks pass
- 5 flaky tests now pass reliably in parallel execution
- Test suite can be run in CI/CD without intermittent failures
- Future developers understand why isolation was needed </success_criteria>
Phase 09.1 Plan 01: Fix Flaky Test Parallel Execution Summary
[Substantive one-liner - what was fixed and how]
Accomplishments
- Identified root cause of 5 flaky tests (state pollution in parallel execution)
- Applied Swift Testing isolation strategy ([
.serializedtrait / proper cleanup / etc.]) - Verified stability with 3 consecutive full suite runs
- Documented isolation reasoning for future maintenance
Files Created/Modified
SportsTimeTests/ScenarioAPlannerSwiftTests.swift- [Describe fix applied]SportsTimeTests/ScenarioBPlannerSwiftTests.swift- [Describe fix applied]SportsTimeTests/ScenarioCPlannerTests.swift- [Describe fix applied, if modified]
Decisions Made
[Document which isolation strategy was chosen and why]
Issues Encountered
[Any challenges during investigation or fix application, or "None"]
Next Phase Readiness
- ✅ Test suite now reliable for CI/CD
- ✅ No intermittent failures
- Ready for Phase 10: Trip Builder Options TDD