Suite6 + P8: Comprehensive task tests + Roborazzi scaffolding
Suite6_ComprehensiveTaskTests ports iOS tests not covered by Suite5/10 (priority/frequency picker variants, custom intervals, completion history, edge cases). Roborazzi screenshot-regression scaffolding in place but gated with @Ignore until pipeline is wired — first `recordRoborazziDebug` run needs manual golden-image review. See docs/screenshot-tests.md for enablement steps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
102
docs/screenshot-tests.md
Normal file
102
docs/screenshot-tests.md
Normal file
@@ -0,0 +1,102 @@
|
||||
# Roborazzi screenshot regression tests (P8)
|
||||
|
||||
Roborazzi is a screenshot-diff testing tool purpose-built for Jetpack /
|
||||
Compose Multiplatform. It runs on the Robolectric-backed JVM unit-test
|
||||
classpath, so no emulator or physical device is required — perfect for
|
||||
CI and for catching UI regressions on every PR.
|
||||
|
||||
## Why screenshot tests?
|
||||
|
||||
Unit tests assert logic; instrumentation tests assert user-visible
|
||||
behaviour. Neither reliably catches *design* regressions: a colour drift
|
||||
in `Theme.kt`, a typography scale change, an accidental padding edit.
|
||||
Screenshot tests close that gap by diffing pixel output against a
|
||||
committed golden set.
|
||||
|
||||
## What we cover
|
||||
|
||||
The initial matrix (see `composeApp/src/androidUnitTest/.../ScreenshotTests.kt`)
|
||||
is intentionally conservative:
|
||||
|
||||
| Surface | Themes | Modes | Total |
|
||||
|---|---|---|---|
|
||||
| Login | Default · Ocean · Midnight | light · dark | 6 |
|
||||
| Tasks | Default · Ocean · Midnight | light · dark | 6 |
|
||||
| Residences | Default · Ocean · Midnight | light · dark | 6 |
|
||||
| Profile | Default · Ocean · Midnight | light · dark | 6 |
|
||||
| Theme palette | Default · Ocean · Midnight | light · dark | 6 |
|
||||
| Complete task | Default · Ocean · Midnight | light · dark | 6 |
|
||||
| **Total** | | | **36** |
|
||||
|
||||
The full 11-theme matrix (132+ images) is deliberately deferred — the
|
||||
cost of reviewer approval on every image outweighs the marginal cover.
|
||||
|
||||
Each test renders a *showcase* composable (pure Material3 primitives)
|
||||
rather than the full production screen. That keeps Roborazzi hermetic:
|
||||
no DataManager, no Ktor client, no ViewModel. A regression in
|
||||
`Theme.kt`'s colour scheme will still surface because the showcases
|
||||
consume every colour slot the real screens use.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Record a fresh golden set (do this on first setup and after intentional UI changes)
|
||||
./gradlew :composeApp:recordRoborazziDebug
|
||||
|
||||
# Verify current UI matches the golden set (fails the build on drift)
|
||||
./gradlew :composeApp:verifyRoborazziDebug
|
||||
|
||||
# Generate side-by-side diff images (useful for review)
|
||||
./gradlew :composeApp:compareRoborazziDebug
|
||||
```
|
||||
|
||||
Output lands under `composeApp/build/outputs/roborazzi/`.
|
||||
|
||||
## Golden-image workflow
|
||||
|
||||
Roborazzi goldens are *not* auto-committed. The workflow is:
|
||||
|
||||
1. Developer changes a composable (intentionally or otherwise).
|
||||
2. CI runs `verifyRoborazziDebug` and fails on any drift.
|
||||
3. Developer inspects the diff locally via `compareRoborazziDebug` or
|
||||
from the CI artifact.
|
||||
4. If the drift is intentional, regenerate via
|
||||
`recordRoborazziDebug` and commit the new PNGs inside the PR so the
|
||||
reviewer explicitly sign-offs on each image change.
|
||||
5. If the drift is a regression, fix the composable and re-run.
|
||||
|
||||
**Reviewer checklist:** every committed `.png` under the roborazzi output
|
||||
dir is an intentional design decision. Scrutinise as carefully as you
|
||||
would scrutinise the code change it accompanies.
|
||||
|
||||
## Adding a new screenshot test
|
||||
|
||||
```kotlin
|
||||
@Test
|
||||
fun mySurface_default_light() = runScreen(
|
||||
name = "my_surface_default_light",
|
||||
theme = AppThemes.Default,
|
||||
darkTheme = false,
|
||||
) {
|
||||
MySurfaceShowcase()
|
||||
}
|
||||
```
|
||||
|
||||
Add the corresponding dark-mode and other-theme variants, then run
|
||||
`recordRoborazziDebug` to generate the initial PNGs.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- Roborazzi requires `@GraphicsMode(Mode.NATIVE)` — the Robolectric
|
||||
version in this repo (4.14.1) supports it.
|
||||
- The test runner uses a fixed device qualifier (`w360dp-h800dp-mdpi`).
|
||||
If you change this, every golden must be regenerated.
|
||||
- `captureRoboImage` only captures the composable tree, not window
|
||||
chrome (status bar, navigation bar). That's intentional — chrome
|
||||
is owned by the OS, not our design system.
|
||||
|
||||
## References
|
||||
|
||||
- Upstream: https://github.com/takahirom/roborazzi
|
||||
- Matrix rationale: see commit message on `P8: Roborazzi screenshot
|
||||
regression test scaffolding`.
|
||||
Reference in New Issue
Block a user