40d2607da8
Suite6_ComprehensiveTaskTests ports iOS tests not covered by Suite5/10 (priority/frequency picker variants, custom intervals, completion history, edge cases). Roborazzi screenshot-regression scaffolding in place but gated with @Ignore until pipeline is wired — first `recordRoborazziDebug` run needs manual golden-image review. See docs/screenshot-tests.md for enablement steps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
103 lines
3.7 KiB
Markdown
103 lines
3.7 KiB
Markdown
# Roborazzi screenshot regression tests (P8)
|
|
|
|
Roborazzi is a screenshot-diff testing tool purpose-built for Jetpack /
|
|
Compose Multiplatform. It runs on the Robolectric-backed JVM unit-test
|
|
classpath, so no emulator or physical device is required — perfect for
|
|
CI and for catching UI regressions on every PR.
|
|
|
|
## Why screenshot tests?
|
|
|
|
Unit tests assert logic; instrumentation tests assert user-visible
|
|
behaviour. Neither reliably catches *design* regressions: a colour drift
|
|
in `Theme.kt`, a typography scale change, an accidental padding edit.
|
|
Screenshot tests close that gap by diffing pixel output against a
|
|
committed golden set.
|
|
|
|
## What we cover
|
|
|
|
The initial matrix (see `composeApp/src/androidUnitTest/.../ScreenshotTests.kt`)
|
|
is intentionally conservative:
|
|
|
|
| Surface | Themes | Modes | Total |
|
|
|---|---|---|---|
|
|
| Login | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Tasks | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Residences | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Profile | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Theme palette | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Complete task | Default · Ocean · Midnight | light · dark | 6 |
|
|
| **Total** | | | **36** |
|
|
|
|
The full 11-theme matrix (132+ images) is deliberately deferred — the
|
|
cost of reviewer approval on every image outweighs the marginal cover.
|
|
|
|
Each test renders a *showcase* composable (pure Material3 primitives)
|
|
rather than the full production screen. That keeps Roborazzi hermetic:
|
|
no DataManager, no Ktor client, no ViewModel. A regression in
|
|
`Theme.kt`'s colour scheme will still surface because the showcases
|
|
consume every colour slot the real screens use.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
# Record a fresh golden set (do this on first setup and after intentional UI changes)
|
|
./gradlew :composeApp:recordRoborazziDebug
|
|
|
|
# Verify current UI matches the golden set (fails the build on drift)
|
|
./gradlew :composeApp:verifyRoborazziDebug
|
|
|
|
# Generate side-by-side diff images (useful for review)
|
|
./gradlew :composeApp:compareRoborazziDebug
|
|
```
|
|
|
|
Output lands under `composeApp/build/outputs/roborazzi/`.
|
|
|
|
## Golden-image workflow
|
|
|
|
Roborazzi goldens are *not* auto-committed. The workflow is:
|
|
|
|
1. Developer changes a composable (intentionally or otherwise).
|
|
2. CI runs `verifyRoborazziDebug` and fails on any drift.
|
|
3. Developer inspects the diff locally via `compareRoborazziDebug` or
|
|
from the CI artifact.
|
|
4. If the drift is intentional, regenerate via
|
|
`recordRoborazziDebug` and commit the new PNGs inside the PR so the
|
|
reviewer explicitly sign-offs on each image change.
|
|
5. If the drift is a regression, fix the composable and re-run.
|
|
|
|
**Reviewer checklist:** every committed `.png` under the roborazzi output
|
|
dir is an intentional design decision. Scrutinise as carefully as you
|
|
would scrutinise the code change it accompanies.
|
|
|
|
## Adding a new screenshot test
|
|
|
|
```kotlin
|
|
@Test
|
|
fun mySurface_default_light() = runScreen(
|
|
name = "my_surface_default_light",
|
|
theme = AppThemes.Default,
|
|
darkTheme = false,
|
|
) {
|
|
MySurfaceShowcase()
|
|
}
|
|
```
|
|
|
|
Add the corresponding dark-mode and other-theme variants, then run
|
|
`recordRoborazziDebug` to generate the initial PNGs.
|
|
|
|
## Known limitations
|
|
|
|
- Roborazzi requires `@GraphicsMode(Mode.NATIVE)` — the Robolectric
|
|
version in this repo (4.14.1) supports it.
|
|
- The test runner uses a fixed device qualifier (`w360dp-h800dp-mdpi`).
|
|
If you change this, every golden must be regenerated.
|
|
- `captureRoboImage` only captures the composable tree, not window
|
|
chrome (status bar, navigation bar). That's intentional — chrome
|
|
is owned by the OS, not our design system.
|
|
|
|
## References
|
|
|
|
- Upstream: https://github.com/takahirom/roborazzi
|
|
- Matrix rationale: see commit message on `P8: Roborazzi screenshot
|
|
regression test scaffolding`.
|