From 102c3484e9ef0a9f6f5a4a27d96a8dfe1e834312 Mon Sep 17 00:00:00 2001 From: admin Date: Thu, 16 Apr 2026 19:36:56 -0500 Subject: [PATCH] docs: export plan + handoff into repo (#1) --- docs/HANDOFF.md | 85 ++++++++++++++++++ docs/PLAN.md | 232 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 317 insertions(+) create mode 100644 docs/HANDOFF.md create mode 100644 docs/PLAN.md diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md new file mode 100644 index 0000000..fcfc54a --- /dev/null +++ b/docs/HANDOFF.md @@ -0,0 +1,85 @@ +# Handoff — Screens VNC app + +A snapshot of project state so another machine (or another Claude Code session) can pick up without any out-of-band context. + +## Resume on a new machine + +```sh +# 1. Clone +git clone git@gitea.treytartt.com:admin/Screens.git +cd Screens + +# 2. Prerequisites (macOS host) +brew install xcodegen # reproducible .xcodeproj generation +# Xcode 16+ required (Xcode 26 tested) + +# 3. Generate the Xcode project (not committed) +xcodegen generate + +# 4. Open in Xcode (or build from CLI) +open Screens.xcodeproj +# - or - +xcodebuild -scheme Screens -destination 'platform=iOS Simulator,name=iPhone 17' build + +# 5. Fast unit tests (no simulator) +cd Packages/VNCCore && swift test # ~0.4s, 3 tests +cd ../VNCUI && swift test # ~3s, 1 test +``` + +## Git layout + +- `main` — cumulative integrated history. Default branch. +- `phase-N-` — per-phase branches; open a PR into `main` and merge. +- Current branch in progress: **`phase-1-vnc-wiring`** (branched from `main` @ Phase 0 scaffold). + +## Phase status + +| Phase | Scope | Status | +|-------|-------|--------| +| 0 — Scaffold | Packages, app target, xcodegen, RoyalVNCKit dep, CI-compilable | ✅ merged to `main` (commit `2cff17f`) | +| 1 — MVP connect/view/tap | Wire `VNCConnectionDelegate`, render framebuffer, touch input, Keychain auth | 🔄 in progress on `phase-1-vnc-wiring` | +| 2 — Input parity | Trackpad mode, hardware keyboard, pointer, adaptive quality, reconnect | ⏳ not started | +| 3 — Productivity | Clipboard sync, multi-monitor, Apple Pencil, screenshots, view-only, curtain mode | ⏳ not started | +| 4 — Polish & ship | iPad multi-window, CloudKit sync, a11y, privacy manifest, TestFlight prep | ⏳ not started | + +Full plan: [docs/PLAN.md](./PLAN.md). + +## Critical architectural notes + +1. **RoyalVNCKit owns its own networking.** It constructs its own `NWConnection` internally from `VNCConnection.Settings`. Our `Transport` protocol in `VNCCore` is *not* on the RFB path — it's kept as the extension point for future SSH tunneling. Don't try to pipe bytes from `DirectTransport` into `VNCConnection`. + +2. **Delegate model.** `SessionController` should conform to `VNCConnectionDelegate` and bridge delegate callbacks (which arrive on internal RoyalVNCKit queues) to `@MainActor` state. Password supplied via `credentialFor:completion:` callback. + +3. **iOS framebuffer rendering is on us.** RoyalVNCKit ships macOS-only `VNCFramebufferView` (NSView). We render by grabbing `framebuffer.cgImage` and setting `FramebufferUIView.contentLayer.contents` on each `didUpdateFramebuffer` callback. IOSurface-backed path available for zero-copy later. + +4. **Swift 6 strict concurrency is on.** Cross-actor hops need care. Use `@MainActor` on everything UI-adjacent; mark Transport-style types as actors. + +5. **Name "Screens" is owned by Edovia.** Pick a different App Store title before any public artifact. + +## Dependencies + +| Package | Purpose | Version | License | +|---------|---------|---------|---------| +| [royalvnc](https://github.com/royalapplications/royalvnc) | RFB protocol, encodings, auth | `branch: main` (tagged releases blocked by transitive CryptoSwift unstable-version constraint) | MIT | +| Apple first-party | Network, SwiftData, Security, UIKit, SwiftUI, Observation | iOS 18 SDK | — | + +## Build artifacts + +- `Screens.xcodeproj/` — **git-ignored**. Regenerate with `xcodegen generate`. +- `Packages/*/.build/` — git-ignored. `swift build` or Xcode resolves. +- `Packages/*/Package.resolved` — git-ignored; change if you want reproducible dep versions across machines (recommended for an app target — consider flipping later). + +## Files to look at first + +- `docs/PLAN.md` — full plan +- `Project.yml` — xcodegen project definition +- `Screens/App/AppStateController.swift` — app-level state machine +- `Packages/VNCCore/Sources/VNCCore/Session/SessionController.swift` — the stub that Phase 1 will replace with a real `VNCConnectionDelegate` integration +- `Packages/VNCUI/Sources/VNCUI/Session/FramebufferUIView.swift` — where incoming framebuffer `CGImage`s land + +## Open decisions + +- **App Store name** — not "Screens" (trademarked). +- **Bundle identifier** — currently `com.example.screens` placeholder; set to your real prefix. +- **Team ID / signing** — currently `CODE_SIGN_STYLE: Automatic`; point at your team for device builds. +- **Privacy manifest** — scheduled for Phase 4; will enumerate `UIPasteboard`, `UserDefaults`, `NSPrivacyAccessedAPICategoryFileTimestamp` reasons. diff --git a/docs/PLAN.md b/docs/PLAN.md new file mode 100644 index 0000000..81682cb --- /dev/null +++ b/docs/PLAN.md @@ -0,0 +1,232 @@ +# Plan: VNC Client App (Screens-style) — iPhone + iPad + +## Context + +The user wants to build a VNC remote-desktop app for iPhone and iPad modeled on [Screens by Edovia](https://www.edovia.com/en/screens/), which is the market leader in the category. Screens exposes a premium, polished VNC experience: Bonjour discovery, saved connections, local + SSH tunneled transport, Touch Mode / Trackpad Mode input, clipboard sync, multi-monitor, hardware keyboard + Apple Pencil, and (in the full product) a relay service and file transfer. + +Working directory `/Users/treyt/Desktop/code/Screens` is empty — this is a greenfield build. Scope agreed with the user: + +- **Platforms**: iPhone + iPad (iOS/iPadOS). No Mac / visionOS in phase 1. +- **RFB engine**: [RoyalVNCKit](https://github.com/royalapplications/royalvnc) (MIT, Swift, supports Raw/CopyRect/RRE/CoRRE/Hextile/Zlib/ZRLE/Tight + VNC password + Apple Remote Desktop auth). iOS UIView rendering is flagged "work in progress" upstream, so we own the UIKit framebuffer view. +- **MVP ambition**: "Feature-parity push" — MVP + clipboard sync, multi-monitor, trackpad mode, Apple Pencil. ~2 months. +- **Remote access strategy**: Tailscale. A device on a tailnet is reached by its Tailscale IP or MagicDNS name exactly like a LAN host, and the transport is already WireGuard-encrypted end-to-end. The app does not need to bundle SSH tunneling or a relay — we just connect TCP and let Tailscale handle the network layer. +- **No account system / subscription** in phase 1. Saved connections sync via CloudKit (iCloud) if the user opts in. + +Intended outcome: a shippable, App Store-quality VNC client that a Mac/Linux/RPi user can drive from an iPhone or iPad on their LAN or through SSH. + +## Architecture + +``` +┌──────────────────────────────────────────────────────────────┐ +│ App target (thin shell, ~10% of code) │ +│ VNCApp.swift · RootView · AppStateController │ +└──────────────────────┬───────────────────────────────────────┘ + │ environment +┌──────────────────────▼───────────────────────────────────────┐ +│ VNCUI (Swift package) │ +│ ConnectionListView · AddConnectionView · SessionView │ +│ FramebufferView (UIViewRepresentable → FramebufferUIView) │ +│ InputMapper (Touch/Trackpad/HW keyboard/Pencil) │ +└──────────────────────┬───────────────────────────────────────┘ + │ +┌──────────────────────▼───────────────────────────────────────┐ +│ VNCCore (Swift package, @testable without simulator) │ +│ SessionController ◀─────── Transport protocol │ +│ │ └── DirectTransport (NW) │ +│ ▼ │ +│ RoyalVNCKit · DiscoveryService (NWBrowser _rfb._tcp) │ +│ ConnectionStore (SwiftData + Keychain refs) │ +│ KeychainService · ClipboardBridge │ +└──────────────────────────────────────────────────────────────┘ +``` + +`Transport` stays behind a protocol so a future relay or optional SSH-tunnel add-on can slot in without touching `SessionController`. + +### Why this shape + +- **Thin `@main`** and `AppStateController`: the app has distinct states (launching → list → connecting → connected → error). Modeling them as an enum avoids "boolean soup" and keeps logic testable. Pattern from the `axiom:axiom-app-composition` skill (Part 1). +- **VNCCore is a Swift Package** with no UI imports. This lets us run the protocol + transport tests with `swift test` (~sub-second) instead of `xcodebuild test`. 60× faster TDD loop. +- **Transport protocol** isolates the network layer from the session layer. Phase-1 has one implementation (`DirectTransport` over `NWConnection`), which already handles both LAN and Tailscale — a tailnet host resolves and routes like any other host from the app's perspective. If we ever add SSH tunneling or a rendezvous relay, they drop in as additional `Transport` conformers. +- **UIKit-backed framebuffer view** wrapped in `UIViewRepresentable`. SwiftUI's `Canvas` is too slow for 30+ fps blits at ~2M pixels. A `CALayer` with `contents = IOSurface`-backed `CGImage` (or a small Metal view if ZRLE/Tight decode runs on GPU later) gives us consistent 60 Hz. + +## Key Dependencies + +| Package | Purpose | License | +| --- | --- | --- | +| [royalvnc](https://github.com/royalapplications/royalvnc) | RFB protocol, encodings, auth | MIT | +| Apple first-party: `Network`, `SwiftData`, `CloudKit`, `Security` (Keychain), `UIKit`, `SwiftUI`, `Combine` | n/a | n/a | + +Tailscale requires no SDK integration — it runs as a system-wide VPN/Network Extension installed separately by the user. Once active, remote hosts are reachable by Tailscale IP or MagicDNS name from our plain `NWConnection`. + +Deployment target: **iOS 18.0**. Gets us `@Observable` macro, SwiftData stability fixes, and modern Network.framework APIs. Skip iOS 26-only features (Liquid Glass components) behind `if #available` gates so we don't block on Xcode 26 availability. + +## Data Model + +SwiftData models live in `VNCCore`: + +```swift +@Model final class SavedConnection { + @Attribute(.unique) var id: UUID + var displayName: String + var host: String // hostname, IPv4, or IPv6 + var port: Int // default 5900 + var colorTag: ColorTag // UI identification + var lastConnectedAt: Date? + var preferredEncodings: [String] // ordered, persisted for reconnect + var keychainTag: String // opaque ref to VNC password entry + var quality: QualityPreset // .adaptive / .high / .low + var viewOnly: Bool + var curtainMode: Bool +} +``` + +VNC passwords go to Keychain (`kSecClassGenericPassword` with `kSecAttrAccessible = kSecAttrAccessibleWhenUnlockedThisDeviceOnly`). The `keychainTag` on the model is the account name — we never persist the secret in SwiftData. Optional CloudKit sync of `SavedConnection` (minus secrets) via SwiftData's CloudKit integration. + +## Phased Delivery + +### Phase 0 — Scaffold (~3 days) +- Create Xcode project (working name — App Store name TBD to avoid trademark conflict; brainstorm candidates before public launch). +- SPM packages `VNCCore` and `VNCUI` inside the repo. +- Add RoyalVNCKit as a dependency. +- Thin `@main`, `AppStateController` with `case launching / list / connecting(SavedConnection) / session(SessionController) / error(AppError)`, `RootView` switch. +- CI via `xcodebuild test` on `VNCUI` + `swift test` on `VNCCore`. +- App icon + launch screen placeholder. + +### Phase 1 — MVP connect + view + tap (~2 weeks) +1. `DiscoveryService` using `NWBrowser` on `_rfb._tcp` and `_workstation._tcp` (macOS Screen Sharing advertises the latter). +2. `ConnectionStore` — SwiftData CRUD + Keychain round-trip. +3. `ConnectionListView`, `AddConnectionView`, basic Liquid Glass cards when available. +4. `DirectTransport` using `NWConnection` (TCP, TLS optional for VeNCrypt later). +5. `SessionController` wrapping `RoyalVNCKit.VNCConnection`: lifecycle, framebuffer updates, reconnection. +6. `FramebufferUIView` (UIKit): `CALayer` whose `contents` is a `CGImage` built from the RoyalVNC framebuffer bytes. Handle partial rect updates (dirty-rect blits, not full redraws). +7. Touch Mode input: tap → left click, two-finger tap → right click, pan → scroll. Nothing fancy yet. +8. VNC password auth + "save to Keychain" flow. + +**Exit criterion**: from a cold launch I can discover my Mac via Bonjour, enter the Screen Sharing password, see the desktop, click a Finder icon, and type into Spotlight. + +### Phase 2 — Input parity (~1.5 weeks) +- Trackpad Mode: a floating soft cursor SwiftUI overlay; pan moves the cursor, tap-to-click, pinch to zoom into the framebuffer. Momentum scroll. +- Hardware keyboard via `pressesBegan`/`pressesEnded` on the framebuffer view — full modifier + function key support (⌘, ⌥, ⌃, ⇧, F1–F20, arrows). +- Magic Trackpad 2 / Magic Mouse: `UIPointerInteraction` + indirect pointer events → VNC pointer events. +- Adaptive quality: choose encoding (Tight vs ZRLE) and JPEG quality based on `NWPathMonitor` link type + measured RTT. +- Reconnect backoff with jitter. + +### Phase 3 — Productivity features (~2 weeks) +- Clipboard sync: bidirectional `UIPasteboard` ↔ VNC `ClientCutText`/`ServerCutText`. Handle Unicode (ISO-8859-1 limitation of classic spec — prefer extended clipboard pseudo-encoding where the server advertises it). Opt-in toggle per connection and a global kill switch for privacy. +- Multi-monitor: parse `DesktopSize` / `ExtendedDesktopSize` pseudo-encodings; offer a monitor picker that sends `SetDesktopSize` or pans the viewport. Remember last selection per connection. +- Apple Pencil: treat as pointer with pressure — if the remote is a Mac running a drawing app, pass pressure via a custom extension if available, otherwise plain pointer. Hover support on M2 iPad Pro. +- Screenshot capture: grab current framebuffer bytes → `CGImage` → share sheet. +- "On disconnect" actions: prompt / reconnect / return to list. +- View Only and Curtain Mode toggles wired into RoyalVNC's session options. + +### Phase 4 — Polish + ship (~1.5 weeks) +- iPad multi-window: `WindowGroup(for: SavedConnection.ID.self)` so each connection can open in its own window, each with its own `SessionController` instance. +- CloudKit sync of `SavedConnection` records via SwiftData's `ModelConfiguration(cloudKitDatabase: .private("iCloud..."))`. Secrets stay local-Keychain only. +- Accessibility pass (VoiceOver, Dynamic Type on non-framebuffer chrome, Reduce Motion). +- Empty states, error recovery, network-change handling (Wi-Fi → cellular with session pause/resume). +- Privacy manifest, App Store screenshots, TestFlight. + +## Critical Files to Create + +``` +Screens.xcodeproj +Screens/ + App/ + VNCApp.swift # @main (thin) + RootView.swift + AppStateController.swift + Resources/ + Assets.xcassets + Info.plist + +Packages/VNCCore/ + Package.swift + Sources/VNCCore/ + Session/ + SessionController.swift # wraps RoyalVNCKit.VNCConnection + SessionState.swift # enum of connection states + Transport/ + Transport.swift # protocol + DirectTransport.swift # NWConnection + SSHTunnelTransport.swift # SwiftNIOSSH + Discovery/ + DiscoveryService.swift # NWBrowser on _rfb._tcp + _workstation._tcp + Storage/ + SavedConnection.swift # @Model + ConnectionStore.swift # actor-wrapped ModelContext helper + Security/ + KeychainService.swift # SecItem wrapper + Clipboard/ + ClipboardBridge.swift + Tests/VNCCoreTests/ + TransportTests.swift + SessionControllerTests.swift # mock Transport + ConnectionStoreTests.swift + +Packages/VNCUI/ + Package.swift + Sources/VNCUI/ + List/ + ConnectionListView.swift + ConnectionCard.swift + Edit/ + AddConnectionView.swift + Session/ + SessionView.swift # hosts the framebuffer + toolbars + FramebufferView.swift # UIViewRepresentable + FramebufferUIView.swift # UIKit CALayer-backed renderer + InputMapper.swift # touch/trackpad/keyboard → RFB events + TrackpadCursorOverlay.swift + ToolbarView.swift + Settings/ + SettingsView.swift +``` + +## Reusable Pieces (don't rebuild) + +- **RoyalVNCKit** gives us the entire RFB state machine, encodings, auth. We *consume* it; we do not fork. +- **`NWBrowser` + `NWListener`** from Network.framework — no need for third-party Bonjour. +- **`AppStateController` pattern** from `axiom:axiom-app-composition` — copy the boilerplate for loading/list/session/error states. +- **`axiom:axiom-swiftui-nav`** for the iPad split-view connection-list-to-session navigation. +- **`axiom:axiom-uikit-bridging`** for `UIViewRepresentable` coordinator patterns on `FramebufferUIView`. +- **`axiom:axiom-keychain`** for the `SecItem` wrapper. +- **`axiom:axiom-networking`** for `NWConnection` TLS/retry patterns reused inside `DirectTransport`. + +## Verification + +Layered testing, unit-heavy because the simulator is slow. + +1. **VNCCore unit tests (`swift test`, ~sub-second)** + - `TransportTests`: feed a synthetic RFB handshake byte stream through a `MockTransport`, assert `SessionController` transitions. + - `ConnectionStoreTests`: in-memory SwiftData container, verify CRUD + Keychain round-trip (Keychain stubbed via protocol in tests). + - `DiscoveryServiceTests`: inject a fake `NWBrowser` surface, verify published result list. + +2. **VNCUI UI snapshot / interaction tests (`xcodebuild test`)** + - `InputMapperTests`: given a `CGPoint` in framebuffer coordinates and a gesture, assert the emitted RFB pointer/key events. + - Preview-driven manual checks for `ConnectionListView`, `AddConnectionView` on iPhone + iPad size classes. + +3. **End-to-end manual matrix** (run before TestFlight each phase): + - Mac (macOS Screen Sharing, port 5900) — discovery, password, Touch Mode click through, clipboard. + - Linux (`tigervnc-server` in Docker) — ZRLE encoding, resolution change during session. + - Raspberry Pi (`realvnc-vnc-server`) — Raspberry Pi Connect auth variant, low-bandwidth adaptive quality. + - Tailscale remote case: the same three above with the iPhone on cellular reaching the host by its MagicDNS name. + - iPad multi-window: two simultaneous sessions to different hosts. + - Cellular → Wi-Fi handoff mid-session (airplane-mode toggle). + +4. **Performance budget** — on iPhone 15, a 2560×1440 @ 30 fps ZRLE stream should stay under 25% CPU and 40% battery-per-hour. Profile with Instruments Time Profiler + Energy Log each phase. See `axiom:axiom-performance-profiling`. + +5. **App Review preflight**: privacy manifest for `NSUserDefaults`, `UIPasteboard`, and network access; screenshots per device class; `axiom:axiom-app-store-submission` checklist before first TestFlight. + +## Open Questions to Resolve Before Phase 0 + +- **App Store name**: "Screens" is owned by Edovia — we cannot ship under that name. Pick a working title (e.g. "Pikelet", "Portal VNC", "Farcast") before public artifacts. +- **Bundle identifier + team ID**: confirm Apple Developer account + agreed bundle prefix. +- **App icon + brand direction**: can be deferred to Phase 5 but worth sketching early. + +## Sources + +- [Screens by Edovia — marketing site](https://www.edovia.com/en/screens/) +- [RoyalVNCKit (royalapplications/royalvnc)](https://github.com/royalapplications/royalvnc) +- [RFB protocol — Wikipedia](https://en.wikipedia.org/wiki/RFB_protocol) · [RFC 6143 (RFB 3.8)](https://datatracker.ietf.org/doc/html/rfc6143) +- [Tailscale on iOS](https://tailscale.com/kb/1020/install-ios) +- Axiom skills consulted: `axiom-app-composition`, `axiom-uikit-bridging`, `axiom-networking`, `axiom-keychain`, `axiom-swiftui-nav`, `axiom-performance-profiling`, `axiom-app-store-submission`.