honeyDueKMP/docs/screenshot-tests.md

Roborazzi screenshot regression tests (P8)

Roborazzi is a screenshot-diff testing tool purpose-built for Jetpack / Compose Multiplatform. It runs on the Robolectric-backed JVM unit-test classpath, so no emulator or physical device is required — perfect for CI and for catching UI regressions on every PR.

Why screenshot tests?

Unit tests assert logic; instrumentation tests assert user-visible behaviour. Neither reliably catches design regressions: a colour drift in Theme.kt, a typography scale change, an accidental padding edit. Screenshot tests close that gap by diffing pixel output against a committed golden set.

What we cover

The initial matrix (see composeApp/src/androidUnitTest/.../ScreenshotTests.kt) is intentionally conservative:

Surface	Themes	Modes	Total
Login	Default · Ocean · Midnight	light · dark	6
Tasks	Default · Ocean · Midnight	light · dark	6
Residences	Default · Ocean · Midnight	light · dark	6
Profile	Default · Ocean · Midnight	light · dark	6
Theme palette	Default · Ocean · Midnight	light · dark	6
Complete task	Default · Ocean · Midnight	light · dark	6
Total			36

The full 11-theme matrix (132+ images) is deliberately deferred — the cost of reviewer approval on every image outweighs the marginal cover.

Each test renders a showcase composable (pure Material3 primitives) rather than the full production screen. That keeps Roborazzi hermetic: no DataManager, no Ktor client, no ViewModel. A regression in Theme.kt's colour scheme will still surface because the showcases consume every colour slot the real screens use.

Commands

# Record a fresh golden set (do this on first setup and after intentional UI changes)
./gradlew :composeApp:recordRoborazziDebug

# Verify current UI matches the golden set (fails the build on drift)
./gradlew :composeApp:verifyRoborazziDebug

# Generate side-by-side diff images (useful for review)
./gradlew :composeApp:compareRoborazziDebug

Output lands under composeApp/build/outputs/roborazzi/.

Golden-image workflow

Roborazzi goldens are not auto-committed. The workflow is:

1. Developer changes a composable (intentionally or otherwise).
2. CI runs verifyRoborazziDebug and fails on any drift.
3. Developer inspects the diff locally via compareRoborazziDebug or from the CI artifact.
4. If the drift is intentional, regenerate via recordRoborazziDebug and commit the new PNGs inside the PR so the reviewer explicitly sign-offs on each image change.
5. If the drift is a regression, fix the composable and re-run.

Reviewer checklist: every committed .png under the roborazzi output dir is an intentional design decision. Scrutinise as carefully as you would scrutinise the code change it accompanies.

Adding a new screenshot test

@Test
fun mySurface_default_light() = runScreen(
    name = "my_surface_default_light",
    theme = AppThemes.Default,
    darkTheme = false,
) {
    MySurfaceShowcase()
}

Add the corresponding dark-mode and other-theme variants, then run recordRoborazziDebug to generate the initial PNGs.

Known limitations

• Roborazzi requires @GraphicsMode(Mode.NATIVE) — the Robolectric version in this repo (4.14.1) supports it.
• The test runner uses a fixed device qualifier (w360dp-h800dp-mdpi). If you change this, every golden must be regenerated.
• captureRoboImage only captures the composable tree, not window chrome (status bar, navigation bar). That's intentional — chrome is owned by the OS, not our design system.

References

• Upstream: https://github.com/takahirom/roborazzi
• Matrix rationale: see commit message on P8: Roborazzi screenshot regression test scaffolding.