Screens/docs/PLAN.md

Plan: VNC Client App (Screens-style) — iPhone + iPad

Context

The user wants to build a VNC remote-desktop app for iPhone and iPad modeled on Screens by Edovia, which is the market leader in the category. Screens exposes a premium, polished VNC experience: Bonjour discovery, saved connections, local + SSH tunneled transport, Touch Mode / Trackpad Mode input, clipboard sync, multi-monitor, hardware keyboard + Apple Pencil, and (in the full product) a relay service and file transfer.

Working directory /Users/treyt/Desktop/code/Screens is empty — this is a greenfield build. Scope agreed with the user:

• Platforms: iPhone + iPad (iOS/iPadOS). No Mac / visionOS in phase 1.
• RFB engine: RoyalVNCKit (MIT, Swift, supports Raw/CopyRect/RRE/CoRRE/Hextile/Zlib/ZRLE/Tight + VNC password + Apple Remote Desktop auth). iOS UIView rendering is flagged "work in progress" upstream, so we own the UIKit framebuffer view.
• MVP ambition: "Feature-parity push" — MVP + clipboard sync, multi-monitor, trackpad mode, Apple Pencil. ~2 months.
• Remote access strategy: Tailscale. A device on a tailnet is reached by its Tailscale IP or MagicDNS name exactly like a LAN host, and the transport is already WireGuard-encrypted end-to-end. The app does not need to bundle SSH tunneling or a relay — we just connect TCP and let Tailscale handle the network layer.
• No account system / subscription in phase 1. Saved connections sync via CloudKit (iCloud) if the user opts in.

Intended outcome: a shippable, App Store-quality VNC client that a Mac/Linux/RPi user can drive from an iPhone or iPad on their LAN or through SSH.

Architecture

┌──────────────────────────────────────────────────────────────┐
│ App target (thin shell, ~10% of code)                        │
│   VNCApp.swift · RootView · AppStateController               │
└──────────────────────┬───────────────────────────────────────┘
                       │ environment
┌──────────────────────▼───────────────────────────────────────┐
│ VNCUI (Swift package)                                        │
│   ConnectionListView · AddConnectionView · SessionView       │
│   FramebufferView (UIViewRepresentable → FramebufferUIView)  │
│   InputMapper (Touch/Trackpad/HW keyboard/Pencil)            │
└──────────────────────┬───────────────────────────────────────┘
                       │
┌──────────────────────▼───────────────────────────────────────┐
│ VNCCore (Swift package, @testable without simulator)         │
│   SessionController ◀─────── Transport protocol              │
│        │                        └── DirectTransport (NW)     │
│        ▼                                                      │
│   RoyalVNCKit  ·  DiscoveryService (NWBrowser _rfb._tcp)     │
│   ConnectionStore (SwiftData + Keychain refs)                │
│   KeychainService · ClipboardBridge                          │
└──────────────────────────────────────────────────────────────┘

Transport stays behind a protocol so a future relay or optional SSH-tunnel add-on can slot in without touching SessionController.

Why this shape

• Thin @main and AppStateController: the app has distinct states (launching → list → connecting → connected → error). Modeling them as an enum avoids "boolean soup" and keeps logic testable. Pattern from the axiom:axiom-app-composition skill (Part 1).
• VNCCore is a Swift Package with no UI imports. This lets us run the protocol + transport tests with swift test (~sub-second) instead of xcodebuild test. 60× faster TDD loop.
• Transport protocol isolates the network layer from the session layer. Phase-1 has one implementation (DirectTransport over NWConnection), which already handles both LAN and Tailscale — a tailnet host resolves and routes like any other host from the app's perspective. If we ever add SSH tunneling or a rendezvous relay, they drop in as additional Transport conformers.
• UIKit-backed framebuffer view wrapped in UIViewRepresentable. SwiftUI's Canvas is too slow for 30+ fps blits at ~2M pixels. A CALayer with contents = IOSurface-backed CGImage (or a small Metal view if ZRLE/Tight decode runs on GPU later) gives us consistent 60 Hz.

Key Dependencies

Package	Purpose	License
royalvnc	RFB protocol, encodings, auth	MIT
Apple first-party: Network, SwiftData, CloudKit, Security (Keychain), UIKit, SwiftUI, Combine	n/a	n/a

Tailscale requires no SDK integration — it runs as a system-wide VPN/Network Extension installed separately by the user. Once active, remote hosts are reachable by Tailscale IP or MagicDNS name from our plain NWConnection.

Deployment target: iOS 18.0. Gets us @Observable macro, SwiftData stability fixes, and modern Network.framework APIs. Skip iOS 26-only features (Liquid Glass components) behind if #available gates so we don't block on Xcode 26 availability.

Data Model

SwiftData models live in VNCCore:

@Model final class SavedConnection {
    @Attribute(.unique) var id: UUID
    var displayName: String
    var host: String                 // hostname, IPv4, or IPv6
    var port: Int                    // default 5900
    var colorTag: ColorTag           // UI identification
    var lastConnectedAt: Date?
    var preferredEncodings: [String] // ordered, persisted for reconnect
    var keychainTag: String          // opaque ref to VNC password entry
    var quality: QualityPreset       // .adaptive / .high / .low
    var viewOnly: Bool
    var curtainMode: Bool
}

VNC passwords go to Keychain (kSecClassGenericPassword with kSecAttrAccessible = kSecAttrAccessibleWhenUnlockedThisDeviceOnly). The keychainTag on the model is the account name — we never persist the secret in SwiftData. Optional CloudKit sync of SavedConnection (minus secrets) via SwiftData's CloudKit integration.

Phased Delivery

Phase 0 — Scaffold (~3 days)

• Create Xcode project (working name — App Store name TBD to avoid trademark conflict; brainstorm candidates before public launch).
• SPM packages VNCCore and VNCUI inside the repo.
• Add RoyalVNCKit as a dependency.
• Thin @main, AppStateController with case launching / list / connecting(SavedConnection) / session(SessionController) / error(AppError), RootView switch.
• CI via xcodebuild test on VNCUI + swift test on VNCCore.
• App icon + launch screen placeholder.

Phase 1 — MVP connect + view + tap (~2 weeks)

1. DiscoveryService using NWBrowser on _rfb._tcp and _workstation._tcp (macOS Screen Sharing advertises the latter).
2. ConnectionStore — SwiftData CRUD + Keychain round-trip.
3. ConnectionListView, AddConnectionView, basic Liquid Glass cards when available.
4. DirectTransport using NWConnection (TCP, TLS optional for VeNCrypt later).
5. SessionController wrapping RoyalVNCKit.VNCConnection: lifecycle, framebuffer updates, reconnection.
6. FramebufferUIView (UIKit): CALayer whose contents is a CGImage built from the RoyalVNC framebuffer bytes. Handle partial rect updates (dirty-rect blits, not full redraws).
7. Touch Mode input: tap → left click, two-finger tap → right click, pan → scroll. Nothing fancy yet.
8. VNC password auth + "save to Keychain" flow.

Exit criterion: from a cold launch I can discover my Mac via Bonjour, enter the Screen Sharing password, see the desktop, click a Finder icon, and type into Spotlight.

Phase 2 — Input parity (~1.5 weeks)

• Trackpad Mode: a floating soft cursor SwiftUI overlay; pan moves the cursor, tap-to-click, pinch to zoom into the framebuffer. Momentum scroll.
• Hardware keyboard via pressesBegan/pressesEnded on the framebuffer view — full modifier + function key support (⌘, ⌥, ⌃, ⇧, F1–F20, arrows).
• Magic Trackpad 2 / Magic Mouse: UIPointerInteraction + indirect pointer events → VNC pointer events.
• Adaptive quality: choose encoding (Tight vs ZRLE) and JPEG quality based on NWPathMonitor link type + measured RTT.
• Reconnect backoff with jitter.

Phase 3 — Productivity features (~2 weeks)

• Clipboard sync: bidirectional UIPasteboard ↔ VNC ClientCutText/ServerCutText. Handle Unicode (ISO-8859-1 limitation of classic spec — prefer extended clipboard pseudo-encoding where the server advertises it). Opt-in toggle per connection and a global kill switch for privacy.
• Multi-monitor: parse DesktopSize / ExtendedDesktopSize pseudo-encodings; offer a monitor picker that sends SetDesktopSize or pans the viewport. Remember last selection per connection.
• Apple Pencil: treat as pointer with pressure — if the remote is a Mac running a drawing app, pass pressure via a custom extension if available, otherwise plain pointer. Hover support on M2 iPad Pro.
• Screenshot capture: grab current framebuffer bytes → CGImage → share sheet.
• "On disconnect" actions: prompt / reconnect / return to list.
• View Only and Curtain Mode toggles wired into RoyalVNC's session options.

Phase 4 — Polish + ship (~1.5 weeks)

• iPad multi-window: WindowGroup(for: SavedConnection.ID.self) so each connection can open in its own window, each with its own SessionController instance.
• CloudKit sync of SavedConnection records via SwiftData's ModelConfiguration(cloudKitDatabase: .private("iCloud...")). Secrets stay local-Keychain only.
• Accessibility pass (VoiceOver, Dynamic Type on non-framebuffer chrome, Reduce Motion).
• Empty states, error recovery, network-change handling (Wi-Fi → cellular with session pause/resume).
• Privacy manifest, App Store screenshots, TestFlight.

Critical Files to Create

Screens.xcodeproj
Screens/
  App/
    VNCApp.swift                       # @main (thin)
    RootView.swift
    AppStateController.swift
  Resources/
    Assets.xcassets
    Info.plist

Packages/VNCCore/
  Package.swift
  Sources/VNCCore/
    Session/
      SessionController.swift          # wraps RoyalVNCKit.VNCConnection
      SessionState.swift               # enum of connection states
    Transport/
      Transport.swift                  # protocol
      DirectTransport.swift            # NWConnection
      SSHTunnelTransport.swift         # SwiftNIOSSH
    Discovery/
      DiscoveryService.swift           # NWBrowser on _rfb._tcp + _workstation._tcp
    Storage/
      SavedConnection.swift            # @Model
      ConnectionStore.swift            # actor-wrapped ModelContext helper
    Security/
      KeychainService.swift            # SecItem wrapper
    Clipboard/
      ClipboardBridge.swift
  Tests/VNCCoreTests/
    TransportTests.swift
    SessionControllerTests.swift       # mock Transport
    ConnectionStoreTests.swift

Packages/VNCUI/
  Package.swift
  Sources/VNCUI/
    List/
      ConnectionListView.swift
      ConnectionCard.swift
    Edit/
      AddConnectionView.swift
    Session/
      SessionView.swift                # hosts the framebuffer + toolbars
      FramebufferView.swift            # UIViewRepresentable
      FramebufferUIView.swift          # UIKit CALayer-backed renderer
      InputMapper.swift                # touch/trackpad/keyboard → RFB events
      TrackpadCursorOverlay.swift
      ToolbarView.swift
    Settings/
      SettingsView.swift

Reusable Pieces (don't rebuild)

• RoyalVNCKit gives us the entire RFB state machine, encodings, auth. We consume it; we do not fork.
• NWBrowser + NWListener from Network.framework — no need for third-party Bonjour.
• AppStateController pattern from axiom:axiom-app-composition — copy the boilerplate for loading/list/session/error states.
• axiom:axiom-swiftui-nav for the iPad split-view connection-list-to-session navigation.
• axiom:axiom-uikit-bridging for UIViewRepresentable coordinator patterns on FramebufferUIView.
• axiom:axiom-keychain for the SecItem wrapper.
• axiom:axiom-networking for NWConnection TLS/retry patterns reused inside DirectTransport.

Verification

Layered testing, unit-heavy because the simulator is slow.

1. VNCCore unit tests (swift test, ~sub-second)

   • TransportTests: feed a synthetic RFB handshake byte stream through a MockTransport, assert SessionController transitions.
   • ConnectionStoreTests: in-memory SwiftData container, verify CRUD + Keychain round-trip (Keychain stubbed via protocol in tests).
   • DiscoveryServiceTests: inject a fake NWBrowser surface, verify published result list.

2. VNCUI UI snapshot / interaction tests (xcodebuild test)

   • InputMapperTests: given a CGPoint in framebuffer coordinates and a gesture, assert the emitted RFB pointer/key events.
   • Preview-driven manual checks for ConnectionListView, AddConnectionView on iPhone + iPad size classes.

3. End-to-end manual matrix (run before TestFlight each phase):

   • Mac (macOS Screen Sharing, port 5900) — discovery, password, Touch Mode click through, clipboard.
   • Linux (tigervnc-server in Docker) — ZRLE encoding, resolution change during session.
   • Raspberry Pi (realvnc-vnc-server) — Raspberry Pi Connect auth variant, low-bandwidth adaptive quality.
   • Tailscale remote case: the same three above with the iPhone on cellular reaching the host by its MagicDNS name.
   • iPad multi-window: two simultaneous sessions to different hosts.
   • Cellular → Wi-Fi handoff mid-session (airplane-mode toggle).

4. Performance budget — on iPhone 15, a 2560×1440 @ 30 fps ZRLE stream should stay under 25% CPU and 40% battery-per-hour. Profile with Instruments Time Profiler + Energy Log each phase. See axiom:axiom-performance-profiling.

5. App Review preflight: privacy manifest for NSUserDefaults, UIPasteboard, and network access; screenshots per device class; axiom:axiom-app-store-submission checklist before first TestFlight.

Open Questions to Resolve Before Phase 0

• App Store name: "Screens" is owned by Edovia — we cannot ship under that name. Pick a working title (e.g. "Pikelet", "Portal VNC", "Farcast") before public artifacts.
• Bundle identifier + team ID: confirm Apple Developer account + agreed bundle prefix.
• App icon + brand direction: can be deferred to Phase 5 but worth sketching early.

Sources

• Screens by Edovia — marketing site
• RoyalVNCKit (royalapplications/royalvnc)
• RFB protocol — Wikipedia · RFC 6143 (RFB 3.8)
• Tailscale on iOS
• Axiom skills consulted: axiom-app-composition, axiom-uikit-bridging, axiom-networking, axiom-keychain, axiom-swiftui-nav, axiom-performance-profiling, axiom-app-store-submission.