0c554cce6a
Records initial golden set + wires verifyRoborazziDebug into CI. Diffs
uploaded as artifact on failure. ScreenshotTests @Ignore removed.
Root cause of the prior RoboMonitoringInstrumentation:102 failure:
createComposeRule() launches ActivityScenarioRule<ComponentActivity>
which fires a MAIN/LAUNCHER intent, but the merged unit-test manifest
declares androidx.activity.ComponentActivity without a LAUNCHER filter,
so Robolectric's PM returns "Unable to resolve activity for Intent".
Fix: switch to the standalone captureRoboImage(path) { composable }
helper from roborazzi-compose, which registers
RoborazziTransparentActivity with Robolectric's shadow PackageManager
at runtime and bypasses ActivityScenario entirely.
Also pin roborazzi outputDir to src/androidUnitTest/roborazzi so
goldens live in git (not build/) and survive gradle clean.
36 goldens, 540KB total.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
121 lines
4.8 KiB
Markdown
121 lines
4.8 KiB
Markdown
# Roborazzi screenshot regression tests (P8)
|
|
|
|
Roborazzi is a screenshot-diff testing tool purpose-built for Jetpack /
|
|
Compose Multiplatform. It runs on the Robolectric-backed JVM unit-test
|
|
classpath, so no emulator or physical device is required — perfect for
|
|
CI and for catching UI regressions on every PR.
|
|
|
|
## Why screenshot tests?
|
|
|
|
Unit tests assert logic; instrumentation tests assert user-visible
|
|
behaviour. Neither reliably catches *design* regressions: a colour drift
|
|
in `Theme.kt`, a typography scale change, an accidental padding edit.
|
|
Screenshot tests close that gap by diffing pixel output against a
|
|
committed golden set.
|
|
|
|
## What we cover
|
|
|
|
The initial matrix (see `composeApp/src/androidUnitTest/.../ScreenshotTests.kt`)
|
|
is intentionally conservative:
|
|
|
|
| Surface | Themes | Modes | Total |
|
|
|---|---|---|---|
|
|
| Login | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Tasks | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Residences | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Profile | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Theme palette | Default · Ocean · Midnight | light · dark | 6 |
|
|
| Complete task | Default · Ocean · Midnight | light · dark | 6 |
|
|
| **Total** | | | **36** |
|
|
|
|
The full 11-theme matrix (132+ images) is deliberately deferred — the
|
|
cost of reviewer approval on every image outweighs the marginal cover.
|
|
|
|
Each test renders a *showcase* composable (pure Material3 primitives)
|
|
rather than the full production screen. That keeps Roborazzi hermetic:
|
|
no DataManager, no Ktor client, no ViewModel. A regression in
|
|
`Theme.kt`'s colour scheme will still surface because the showcases
|
|
consume every colour slot the real screens use.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
# Record a fresh golden set (do this on first setup and after intentional UI changes)
|
|
./gradlew :composeApp:recordRoborazziDebug
|
|
|
|
# Verify current UI matches the golden set (fails the build on drift)
|
|
./gradlew :composeApp:verifyRoborazziDebug
|
|
|
|
# Generate side-by-side diff images (useful for review)
|
|
./gradlew :composeApp:compareRoborazziDebug
|
|
```
|
|
|
|
Committed goldens live at
|
|
`composeApp/src/androidUnitTest/roborazzi/` — pinned there via the
|
|
`roborazzi { outputDir = ... }` block in `composeApp/build.gradle.kts`
|
|
so they survive `gradle clean`. Diffs and intermediate artefacts land
|
|
under `composeApp/build/outputs/roborazzi/` and are uploaded as a CI
|
|
artifact on failure (`android-ui-tests.yml → Upload screenshot diffs
|
|
on failure`).
|
|
|
|
## Golden-image workflow
|
|
|
|
Roborazzi goldens *are* committed alongside the tests — see
|
|
`composeApp/src/androidUnitTest/roborazzi/`. The workflow is:
|
|
|
|
1. Developer changes a composable (intentionally or otherwise).
|
|
2. CI runs `verifyRoborazziDebug` and fails on any drift; the
|
|
`roborazzi-diffs` artifact is uploaded for review.
|
|
3. Developer inspects the diff locally via `compareRoborazziDebug` or
|
|
from the CI artifact.
|
|
4. If the drift is intentional, regenerate via
|
|
`recordRoborazziDebug` and commit the new PNGs inside the PR so the
|
|
reviewer explicitly signs off on each image change.
|
|
5. If the drift is a regression, fix the composable and re-run.
|
|
|
|
**Reviewer checklist:** every committed `.png` under the roborazzi output
|
|
dir is an intentional design decision. Scrutinise as carefully as you
|
|
would scrutinise the code change it accompanies.
|
|
|
|
## Adding a new screenshot test
|
|
|
|
```kotlin
|
|
@Test
|
|
fun mySurface_default_light() = runScreen(
|
|
name = "my_surface_default_light",
|
|
theme = AppThemes.Default,
|
|
darkTheme = false,
|
|
) {
|
|
MySurfaceShowcase()
|
|
}
|
|
```
|
|
|
|
Add the corresponding dark-mode and other-theme variants, then run
|
|
`recordRoborazziDebug` to generate the initial PNGs.
|
|
|
|
## Known limitations
|
|
|
|
- Roborazzi requires `@GraphicsMode(Mode.NATIVE)` — the Robolectric
|
|
version in this repo (4.14.1) supports it.
|
|
- The test runner uses a fixed device qualifier (`w360dp-h800dp-mdpi`).
|
|
If you change this, every golden must be regenerated.
|
|
- `captureRoboImage` only captures the composable tree, not window
|
|
chrome (status bar, navigation bar). That's intentional — chrome
|
|
is owned by the OS, not our design system.
|
|
- We use the standalone `captureRoboImage(filePath) { composable }`
|
|
helper from `roborazzi-compose` instead of the
|
|
`createComposeRule() + RoborazziRule` approach. The helper registers
|
|
`RoborazziTransparentActivity` with Robolectric's shadow PackageManager
|
|
on its own, avoiding the
|
|
"Unable to resolve activity for Intent MAIN/LAUNCHER cmp=.../ComponentActivity"
|
|
failure you hit if `createComposeRule` tries to launch
|
|
`androidx.activity.ComponentActivity` through `ActivityScenario` on the
|
|
unit-test classpath (where the manifest declares `ComponentActivity`
|
|
without a MAIN/LAUNCHER intent filter).
|
|
|
|
## References
|
|
|
|
- Upstream: https://github.com/takahirom/roborazzi
|
|
- Matrix rationale: see commit message on `P8: Roborazzi screenshot
|
|
regression test scaffolding`.
|