docs(09.1): create phase plan
Phase 09.1: Fix Flaky Test When Ran In Parallel - 1 plan created - 2 total tasks defined - Ready for execution Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Trey t
180
.planning/phases/09.1-fix-flaky-test-parallel/09.1-01-PLAN.md
Normal file
180
.planning/phases/09.1-fix-flaky-test-parallel/09.1-01-PLAN.md
Normal file
@@ -0,0 +1,180 @@
|
||||
---
|
||||
phase: 09.1-fix-flaky-test-parallel
|
||||
type: execute
|
||||
---
|
||||
|
||||
<objective>
|
||||
Fix 5 flaky tests that pass individually but fail in parallel execution, blocking CI/CD reliability.
|
||||
|
||||
Purpose: Ensure test suite stability by resolving Swift Testing parallel execution state pollution.
|
||||
Output: All tests pass reliably in full suite runs, enabling consistent CI/CD testing.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
~/.claude/get-shit-done/workflows/execute-phase.md
|
||||
./summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/ROADMAP.md
|
||||
@.planning/STATE.md
|
||||
@.planning/codebase/TESTING.md
|
||||
@.planning/codebase/CONVENTIONS.md
|
||||
|
||||
# Prior Phase Context
|
||||
@.planning/phases/09-trip-planner-modes-tdd/09-01-SUMMARY.md
|
||||
@.planning/phases/09-trip-planner-modes-tdd/09-02-SUMMARY.md
|
||||
@.planning/phases/09-trip-planner-modes-tdd/09-03-SUMMARY.md
|
||||
|
||||
# Affected Test Files
|
||||
@SportsTimeTests/ScenarioAPlannerSwiftTests.swift
|
||||
@SportsTimeTests/ScenarioBPlannerTests.swift
|
||||
@SportsTimeTests/ScenarioCPlannerTests.swift
|
||||
|
||||
**Problem Summary:**
|
||||
5 tests fail when run in full suite but pass individually:
|
||||
1. `plan_StopDepartureDate_IsLastGameDate()` (ScenarioAPlannerSwiftTests)
|
||||
2. `plan_ManyGames_HandledEfficiently()` (ScenarioAPlannerSwiftTests)
|
||||
3. `plan_ThreeSameDayGames_PicksFeasibleCombinations()` (ScenarioAPlannerSwiftTests)
|
||||
4. `plan_FillerSameDayAsAnchor_Excluded()` (ScenarioBPlannerTests)
|
||||
5. `plan_MustSeeGamesTooFarApart_Fails()` (ScenarioBPlannerTests)
|
||||
|
||||
**Root Cause:**
|
||||
Swift Testing parallel execution + simulator state pollution. Tests share mutable state or depend on execution order.
|
||||
|
||||
**Tech Stack Available:**
|
||||
- Swift Testing framework (iOS 26+)
|
||||
- `.serialized` trait for sequential execution
|
||||
- Test isolation via setup/teardown
|
||||
- Per-test state management
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Investigate root cause and apply Swift Testing isolation</name>
|
||||
<files>SportsTimeTests/ScenarioAPlannerSwiftTests.swift, SportsTimeTests/ScenarioBPlannerTests.swift, SportsTimeTests/ScenarioCPlannerTests.swift</files>
|
||||
<action>
|
||||
Run full test suite to reproduce failures and confirm which tests fail. Then investigate state pollution:
|
||||
|
||||
1. **Check for shared mutable state** - Look for static properties, shared actors, or global state in test suites
|
||||
2. **Check for execution order dependencies** - Look for tests that depend on previous test side effects
|
||||
3. **Check for async cleanup issues** - Look for incomplete async operations or lingering tasks
|
||||
|
||||
Apply Swift Testing isolation patterns:
|
||||
|
||||
**Option A: Add `.serialized` trait** (preferred for quick fix):
|
||||
```swift
|
||||
@Suite("ScenarioAPlanner Tests", .serialized)
|
||||
struct ScenarioAPlannerSwiftTests {
|
||||
// Forces sequential execution, prevents parallel state pollution
|
||||
}
|
||||
```
|
||||
|
||||
**Option B: Proper test isolation** (if state pollution identified):
|
||||
- Add per-test setup/teardown to reset shared state
|
||||
- Use `init()` and `deinit` for test suite lifecycle management
|
||||
- Ensure actors and services are created fresh per test
|
||||
|
||||
**What to avoid and WHY:**
|
||||
- Don't just disable parallel testing globally - this slows down the entire suite
|
||||
- Don't skip or comment out failing tests - they validate correct behavior
|
||||
- Don't modify test expectations to make them pass - tests define correctness
|
||||
|
||||
Apply the minimal fix that ensures reliability. If `.serialized` fixes it, use that. If specific state needs cleanup, add proper isolation.
|
||||
</action>
|
||||
<verify>
|
||||
Run full test suite 3 times consecutively:
|
||||
```bash
|
||||
for i in 1 2 3; do
|
||||
echo "Run $i:"
|
||||
xcodebuild -project SportsTime.xcodeproj -scheme SportsTime -destination 'platform=iOS Simulator,name=iPhone 17,OS=26.2' test 2>&1 | grep -E "(Test Suite.*started|Test Suite.*passed|Test Suite.*failed|failed)"
|
||||
done
|
||||
```
|
||||
|
||||
All 5 previously flaky tests must pass in all 3 runs.
|
||||
</verify>
|
||||
<done>All tests pass reliably in parallel execution, no flaky failures across 3 consecutive full suite runs</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Document root cause and solution in test files</name>
|
||||
<files>SportsTimeTests/ScenarioAPlannerSwiftTests.swift, SportsTimeTests/ScenarioBPlannerTests.swift, SportsTimeTests/ScenarioCPlannerTests.swift</files>
|
||||
<action>
|
||||
Add documentation comment to each modified test suite explaining:
|
||||
1. Why the fix was needed (parallel execution state pollution)
|
||||
2. What pattern was applied (`.serialized` trait or isolation strategy)
|
||||
3. When it can be removed (if state pollution is ever eliminated)
|
||||
|
||||
Example:
|
||||
```swift
|
||||
// MARK: - Swift Testing Isolation
|
||||
// `.serialized` trait applied to prevent parallel execution state pollution.
|
||||
// Tests pass individually but fail in parallel due to shared simulator state.
|
||||
// Can be removed if test isolation is improved in future phases.
|
||||
@Suite("ScenarioAPlanner Tests", .serialized)
|
||||
struct ScenarioAPlannerSwiftTests {
|
||||
```
|
||||
|
||||
This ensures future developers understand the tradeoff (sequential vs parallel) and the reasoning.
|
||||
</action>
|
||||
<verify>Documentation comments present in all modified test files</verify>
|
||||
<done>All modified test suites have clear comments explaining the isolation strategy and reasoning</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<verification>
|
||||
Before declaring phase complete:
|
||||
- [ ] Full test suite runs successfully 3 times consecutively
|
||||
- [ ] All 5 previously flaky tests now pass reliably
|
||||
- [ ] No new test failures introduced
|
||||
- [ ] Documentation comments explain isolation strategy
|
||||
- [ ] Git commit created with proper format
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
|
||||
- All tasks completed
|
||||
- All verification checks pass
|
||||
- 5 flaky tests now pass reliably in parallel execution
|
||||
- Test suite can be run in CI/CD without intermittent failures
|
||||
- Future developers understand why isolation was needed
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
After completion, create `.planning/phases/09.1-fix-flaky-test-parallel/09.1-01-SUMMARY.md`:
|
||||
|
||||
# Phase 09.1 Plan 01: Fix Flaky Test Parallel Execution Summary
|
||||
|
||||
**[Substantive one-liner - what was fixed and how]**
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- Identified root cause of 5 flaky tests (state pollution in parallel execution)
|
||||
- Applied Swift Testing isolation strategy ([`.serialized` trait / proper cleanup / etc.])
|
||||
- Verified stability with 3 consecutive full suite runs
|
||||
- Documented isolation reasoning for future maintenance
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `SportsTimeTests/ScenarioAPlannerSwiftTests.swift` - [Describe fix applied]
|
||||
- `SportsTimeTests/ScenarioBPlannerSwiftTests.swift` - [Describe fix applied]
|
||||
- `SportsTimeTests/ScenarioCPlannerTests.swift` - [Describe fix applied, if modified]
|
||||
|
||||
## Decisions Made
|
||||
|
||||
[Document which isolation strategy was chosen and why]
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
[Any challenges during investigation or fix application, or "None"]
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
- ✅ Test suite now reliable for CI/CD
|
||||
- ✅ No intermittent failures
|
||||
- Ready for Phase 10: Trip Builder Options TDD
|
||||
|
||||
</output>
|
||||
Reference in New Issue
Block a user