Compare commits

20 Commits

Author SHA1 Message Date
Trey T
845136222d Use Portal as the app icon
Assets.xcassets/AppIcon.appiconset with a 1024x1024 universal entry
(iOS 17+ single-size format; actool thins for each device). Info.plist
gets CFBundleIconName=AppIcon.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 14:36:30 -05:00
Trey T
0f415ab498 Add 5 icon concepts for review
Generated via icons/generate.py. See PHILOSOPHY.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 14:33:13 -05:00
Trey T
c1bed4f53b FramebufferUIView: disable CALayer implicit animations on frame updates
Every time Mac repainted a region we set imageLayer.contents to a new
CGImage. CALayer's default action for .contents is a 0.25s crossfade, so
big repaints (like after a click — cursor + button + window-focus) looked
like a pulse. Tap seemed "flickery"; actually the whole view was doing
quarter-second crossfades constantly, most just weren't big enough to
notice until a chunky repaint hit.

Override imageLayer.actions with NSNull for contents/contentsRect/frame/
transform so blits are instantaneous, and wrap apply() in a
CATransaction.setDisableActions(true) for safety.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 14:21:33 -05:00
Trey T
4ff3e4b030 Session: add a persistent chrome-toggle handle at top center
Three-finger tap still works as a power-user shortcut, but now there's a
small glass chevron pill at the top center that flips between chevron.up
(hide toolbar) and chevron.down (show toolbar) based on chrome state.
Discoverable and reachable from one hand.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 14:14:51 -05:00
Trey T
8177be94a5 FramebufferUIView: give it a full UITextInputTraits implementation
Class-level UIKeyInput conformance without UITextInputTraits means iOS
falls back to default traits — autocorrect on, predictive on, smart
quotes/dashes on. The suggestion engine was swallowing most keystrokes
before insertText() could forward them (the "1 in 6 chars" symptom).

Declaring all the traits as @objc stored properties with permissive
(autocorrect=.no, etc.) values turns every suggestion layer off so each
key tap produces exactly one insertText() and hits the remote.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 14:05:13 -05:00
Trey T
0f1d2fa6f6 The actual bug: inputMode:.none silently drops every pointer/key event
VNCConnection.Settings.inputMode looks like it configures keyboard-shortcut
forwarding on macOS, but in RoyalVNCKit's enqueue path it's a master gate:

    // VNCConnection+Queue.swift
    guard settings.inputMode != .none else { return }   // every path

So every PointerEvent and KeyEvent we enqueued was discarded before hitting
the wire. The Mac received zero input even though the framebuffer was live.
Frames streamed because that queue is server→client, not gated by inputMode.

Fix: pass .forwardKeyboardShortcutsEvenIfInUseLocally. On iOS we have no
local keyboard shortcuts to steal from, so the most permissive value is
safe and it unblocks the input queue.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 13:42:48 -05:00
Trey T
a21946ba2c SessionView: surface a "View only" badge in the chrome
When a connection is configured with viewOnly=true, the controller silently
drops every key, pointer, and scroll event. Without a visible indicator the
behavior looks like the keyboard is broken. Yellow capsule next to the
connection label makes it obvious.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 23:17:10 -05:00
Trey T
4408bca53b UI test: prefer real on-screen keyboard taps
Drives the test through app.keyboards.firstMatch.keys[…].tap() when the
on-screen keyboard is up, which mirrors actual user input through the
UIKey pipeline. Falls back to app.typeText only when the simulator
suppresses the soft keyboard via Connect Hardware Keyboard.

Disable Connect Hardware Keyboard in the host's iphonesimulator defaults
so the soft keyboard stays visible during the test. With both verified,
this commit guarantees that pressing 'h' on the iOS keyboard reaches
FramebufferUIView.insertText("h").

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 23:12:29 -05:00
Trey T
0e25dbeba4 Fix the keyboard typing pipeline + add a test that catches the regression
The earlier UIKeyInput conformance was declared in a separate extension. ObjC
protocol conformance via Swift extension is fragile when the protocol
inherits another @objc protocol (UIKeyInput inherits UITextInputTraits) — the
runtime didn't always pick up insertText:, so the on-screen keyboard came
up but characters never reached controller.type(_:).

Fix: declare UIKeyInput conformance directly on FramebufferUIView's class
declaration, with insertText / deleteBackward / hasText as native members.

Also caught and fixed by the new UI test:
- The toolbar's keyboard-icon button had a 20×13 hit region (SF Symbol size)
  even though the visual frame was 34×34 — XCUI taps couldn't land on it
  reliably. .contentShape(Rectangle()) widens the hit area to the frame.
- accessibilityValue is reserved by iOS for UIKeyInput-classed views (treats
  them as TextView), so a separate hidden "fb-diag" accessibility probe
  records keyboard plumbing events for the test to verify.

Tests:
- KeyboardInputTests (5): pure mapping from String → X11 keysym down/up pairs
- ScreensUITests.testSoftwareKeyboardSendsCharactersToFramebuffer:
  opens a session, taps the keyboard toggle, types "hi" via the system
  keyboard, and asserts the framebuffer's diagnostic probe records
  [ins:h] and [ins:i] — proving the chars reach controller.type(_:)
- A SwiftUI state probe (sessionview-state) verifies the binding flips,
  which guards against future tap-routing regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 23:04:03 -05:00
Trey T
da882531d1 Make the keyboard button actually present a keyboard
The toolbar's keyboard icon used to toggle a custom function-key bar — it
never opened the iOS system keyboard, so users couldn't type into the remote.

Fix: FramebufferUIView now conforms to UIKeyInput + UITextInputTraits, so
becoming first responder presents the iOS keyboard. Tapping the keyboard
button toggles software-keyboard visibility on the framebuffer view via a
SwiftUI binding. While the keyboard is up, an inputAccessoryView toolbar
(esc / tab / ctrl / ⌘ / ⌥ / ←↓↑→ / dismiss) sits directly above it, with
each button forwarded to the existing controller.send… APIs.

The standalone SoftKeyboardBar overlay is no longer used.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 22:34:28 -05:00
Trey T
689e30d59a Support Apple Remote Desktop auth via account username+password
RoyalVNCKit prioritizes .diffieHellman (ARD) over .vnc during handshake when
both are offered. My delegate adapter was passing an empty username to
VNCUsernamePasswordCredential, so any Mac with user-account screen sharing
enabled rejected the credential before .vnc fallback could happen.

Fix: persist a username on SavedConnection and pipe it through to the
credential callback. Leave blank to use the VNC-only password path.

AddConnection footer now explains the two Mac paths:
  • User account (ARD) — macOS short name + full account password
  • VNC-only password — blank username + ≤8 char password

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 22:25:27 -05:00
Trey T
497b8a42be Add UI test covering layout + core flows
Verifies:
- Title lives in the top 25% of the screen (guards against letterboxing regressions)
- Plus button opens Add Connection sheet; Cancel dismisses
- Settings gear opens Settings sheet; Done dismisses
- Search field accepts input and keeps the top chrome visible
- Empty-state CTA routes to Add Connection

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 21:59:30 -05:00
Trey T
876d08cbf3 Add UILaunchScreen — fixes screen letterboxing
Without UILaunchScreen (or UILaunchStoryboardName) in Info.plist, iOS runs
the app in legacy device-scaling mode and letterboxes it inside a smaller
iPhone-sized window. That's where the unexplained black bars at the top and
bottom of every screenshot came from — the SwiftUI layout was correct all
along, but iOS was rendering the entire app inside a ~75% viewport.

Adding an empty UILaunchScreen dict opts into modern full-screen rendering.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 21:53:23 -05:00
Trey T
6b50184bcc Kill the NavigationStack on the list — that was the floating-chrome bug
iOS 26's NavigationStack reserves space for its floating Liquid Glass nav bar
even with `.toolbar(.hidden, for: .navigationBar)` — that's why "Screens"
landed ~180pt down from the dynamic island with a wedge of black above it.

Removed the NavigationStack from the list entirely. Layout is now a plain
top-anchored VStack: top chrome (gear + Screens + plus + search) flush with
the safe-area top, then a ScrollView with LazyVStack of cards filling the
rest of the screen. Sessions present via fullScreenCover instead of
NavigationLink, so we don't need NavigationStack here at all.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 21:33:50 -05:00
Trey T
fcdd19ceb9 List screen: ditch nav bar, build flush top chrome
The system nav bar floated low on iOS 26 and left a wedge of black above the
title. Replaced it with a custom top row pinned to the safe-area top: gear ⟶
big "Screens" wordmark ⟶ +. Search bar lives directly beneath, sections start
right after — no centered-in-the-void layout. Background gets a subtle blue
radial bloom so the floating glass buttons have something to anchor to.

Saved-empty state is now a glass card with an icon and a gradient CTA button.
Connection rows are full-width glass cards with rounded corners; long-press
gives Edit / Open in New Window / Delete.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 21:19:41 -05:00
Trey T
333c08724f Redesign UI for iOS 26 Liquid Glass
- Fix port-formatting bug: Int interpolation was adding a locale grouping
  separator ("5,900"); now renders "5900" via portLabel helper.
- LiquidGlass helpers: glassSurface/interactiveGlassSurface/glassButton wrap
  iOS 26's .glassEffect / .buttonStyle(.glass) / scrollEdgeEffectStyle with
  iOS 18 fallbacks (ultraThinMaterial + stroke) gated by #available.
- List: searchable, labeled Bonjour section with "looking for computers"
  state, empty-state CTA, hover-ready rounded discovery buttons, subtle
  dark gradient background, connection cards with color swatch +
  monospaced host:port and chevron.
- Session: floating glass back-pill + connection-status pill + toolbar
  capsule; three-finger tap toggles chrome; disconnect dialog upgraded to
  a 28pt glass card with role-based glyphs/tints.
- Soft keyboard bar redesigned as a rounded glass panel with pill keys.
- Add/Edit form: horizontal color-tag picker, show/hide password eye,
  helpful footers (Tailscale hint, 8-char VNC-password reminder,
  View-only explainer).
- Settings: app-icon-style hero, grouped sections with footers, links to
  privacy policy and RoyalVNCKit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 21:04:17 -05:00
Trey T
8e01068ad3 Add edit-connection flow
Edit reuses AddConnectionView with an `editing:` parameter that prefills the
form and updates in-place; password field becomes optional ("leave blank to
keep current"). Surfaced via context menu and a leading-edge swipe action on
each saved row.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 20:21:14 -05:00
Trey T
fcad267493 Set bundle id to com.tt.screens (dev + prod)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 20:08:50 -05:00
Trey T
1c01b3573f Phases 1-4: full VNC client implementation
- SessionController wraps RoyalVNCKit.VNCConnection via nonisolated delegate
  adapter that bridges callbacks to @MainActor; Keychain-resolved passwords;
  reconnect with jittered exponential backoff; NWPathMonitor adaptive-quality
  hook; framebuffer rendered to CALayer.contents from didUpdateFramebuffer.
- Touch + trackpad input modes with floating soft cursor overlay; hardware
  keyboard via pressesBegan/Ended → X11 keysyms; UIPointerInteraction with
  hidden cursor for indirect pointers; pinch-to-zoom; Apple Pencil as direct
  touch; two-finger pan / indirect scroll wheel events.
- Bidirectional clipboard sync (per-connection opt-in); multi-monitor screen
  picker with input remapping; screenshot capture → share sheet; on-disconnect
  reconnect/close prompt; view-only and curtain-mode persisted.
- iPad multi-window via WindowGroup(for: UUID.self) + context-menu open;
  CloudKit-backed ModelContainer with local fallback; PrivacyInfo.xcprivacy.

10 VNCCore tests + 4 VNCUI tests pass; iPhone and iPad simulator builds clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 20:07:54 -05:00
102c3484e9 docs: export plan + handoff into repo (#1) 2026-04-16 19:36:56 -05:00
44 changed files with 4471 additions and 209 deletions

View File

@@ -1,26 +1,25 @@
import Foundation
import Observation
#if canImport(UIKit)
import UIKit
#endif
@Observable
@MainActor
public final class ClipboardBridge {
public var isEnabled: Bool = true
public var isEnabled: Bool
public init() {}
public init(isEnabled: Bool = true) {
self.isEnabled = isEnabled
}
public func readLocal() -> String? {
#if canImport(UIKit)
return UIPasteboard.general.string
#else
return nil
#endif
guard isEnabled else { return nil }
return ClipboardSink.read()
}
public func writeLocal(_ string: String) {
public func writeLocal(_ text: String) {
guard isEnabled else { return }
#if canImport(UIKit)
UIPasteboard.general.string = string
#endif
ClipboardSink.set(text)
}
}

View File

@@ -3,11 +3,64 @@ import Network
import Observation
public struct DiscoveredHost: Identifiable, Hashable, Sendable {
public let id: String // stable identifier (name + type)
public let id: String
public let displayName: String
public let serviceType: String
public let host: String?
public let port: Int?
public let serviceType: String
fileprivate let endpoint: BonjourEndpointReference?
public init(
id: String,
displayName: String,
serviceType: String,
host: String? = nil,
port: Int? = nil
) {
self.id = id
self.displayName = displayName
self.serviceType = serviceType
self.host = host
self.port = port
self.endpoint = nil
}
fileprivate init(
id: String,
displayName: String,
serviceType: String,
endpoint: BonjourEndpointReference?
) {
self.id = id
self.displayName = displayName
self.serviceType = serviceType
self.host = nil
self.port = nil
self.endpoint = endpoint
}
}
fileprivate struct BonjourEndpointReference: Hashable, Sendable {
let endpoint: NWEndpoint
static func == (lhs: BonjourEndpointReference, rhs: BonjourEndpointReference) -> Bool {
lhs.endpoint.debugDescription == rhs.endpoint.debugDescription
}
func hash(into hasher: inout Hasher) {
hasher.combine(endpoint.debugDescription)
}
}
public struct ResolvedHost: Sendable, Hashable {
public let host: String
public let port: Int
}
public enum DiscoveryError: Error, Sendable {
case unresolvable
case timedOut
case unsupported
}
@Observable
@@ -25,12 +78,13 @@ public final class DiscoveryService {
isBrowsing = true
hosts = []
for type in ["_rfb._tcp", "_workstation._tcp"] {
for type in DiscoveryService.serviceTypes {
let descriptor = NWBrowser.Descriptor.bonjour(type: type, domain: nil)
let browser = NWBrowser(for: descriptor, using: .tcp)
browser.browseResultsChangedHandler = { [weak self] results, _ in
let snapshot = results
Task { @MainActor in
self?.merge(results: results, serviceType: type)
self?.merge(results: snapshot, serviceType: type)
}
}
browser.start(queue: .main)
@@ -44,19 +98,100 @@ public final class DiscoveryService {
isBrowsing = false
}
public func resolve(_ host: DiscoveredHost) async throws -> ResolvedHost {
guard let reference = host.endpoint else {
if let h = host.host, let p = host.port {
return ResolvedHost(host: h, port: p)
}
throw DiscoveryError.unsupported
}
return try await Self.resolve(endpoint: reference.endpoint,
defaultPort: host.serviceType == "_rfb._tcp" ? 5900 : 5900)
}
nonisolated static let serviceTypes = ["_rfb._tcp", "_workstation._tcp"]
private func merge(results: Set<NWBrowser.Result>, serviceType: String) {
let newHosts = results.compactMap { result -> DiscoveredHost? in
guard case let .service(name, type, _, _) = result.endpoint else { return nil }
let id = "\(type)\(name)"
return DiscoveredHost(
id: "\(type)\(name)",
id: id,
displayName: name,
host: nil, // resolved at connect time
port: nil,
serviceType: serviceType
serviceType: serviceType,
endpoint: BonjourEndpointReference(endpoint: result.endpoint)
)
}
var merged = hosts.filter { $0.serviceType != serviceType }
merged.append(contentsOf: newHosts)
hosts = merged.sorted { $0.displayName < $1.displayName }
}
nonisolated private static func resolve(endpoint: NWEndpoint,
defaultPort: Int) async throws -> ResolvedHost {
let parameters = NWParameters.tcp
let connection = NWConnection(to: endpoint, using: parameters)
let resumeBox = ResumeBox()
return try await withCheckedThrowingContinuation { continuation in
connection.stateUpdateHandler = { state in
switch state {
case .ready:
if let endpoint = connection.currentPath?.remoteEndpoint,
let resolvedHost = Self.parseEndpoint(endpoint, defaultPort: defaultPort) {
if resumeBox.tryClaim() {
connection.cancel()
continuation.resume(returning: resolvedHost)
}
} else {
if resumeBox.tryClaim() {
connection.cancel()
continuation.resume(throwing: DiscoveryError.unresolvable)
}
}
case .failed(let error):
if resumeBox.tryClaim() {
connection.cancel()
continuation.resume(throwing: error)
}
case .cancelled:
if resumeBox.tryClaim() {
continuation.resume(throwing: CancellationError())
}
default:
break
}
}
connection.start(queue: .global(qos: .userInitiated))
}
}
private final class ResumeBox: @unchecked Sendable {
private let lock = NSLock()
private var resumed = false
func tryClaim() -> Bool {
lock.lock()
defer { lock.unlock() }
guard !resumed else { return false }
resumed = true
return true
}
}
nonisolated private static func parseEndpoint(_ endpoint: NWEndpoint,
defaultPort: Int) -> ResolvedHost? {
switch endpoint {
case .hostPort(let host, let port):
let hostString: String
switch host {
case .name(let name, _): hostString = name
case .ipv4(let addr): hostString = "\(addr)"
case .ipv6(let addr): hostString = "\(addr)"
@unknown default: return nil
}
return ResolvedHost(host: hostString, port: Int(port.rawValue))
default:
return nil
}
}
}

View File

@@ -0,0 +1,97 @@
import Foundation
import Network
public struct NetworkPathSnapshot: Sendable, Equatable {
public enum LinkType: Sendable, Hashable {
case wifi, cellular, wired, loopback, other, unavailable
}
public let isAvailable: Bool
public let isExpensive: Bool
public let isConstrained: Bool
public let link: LinkType
public init(isAvailable: Bool,
isExpensive: Bool,
isConstrained: Bool,
link: LinkType) {
self.isAvailable = isAvailable
self.isExpensive = isExpensive
self.isConstrained = isConstrained
self.link = link
}
public static let unknown = NetworkPathSnapshot(
isAvailable: true,
isExpensive: false,
isConstrained: false,
link: .other
)
}
public protocol NetworkPathProviding: Sendable {
var pathChanges: AsyncStream<NetworkPathSnapshot> { get }
}
public final class NWPathObserver: NetworkPathProviding, @unchecked Sendable {
private let monitor: NWPathMonitor
private let queue = DispatchQueue(label: "com.screens.vnc.path")
public init() {
self.monitor = NWPathMonitor()
}
public var pathChanges: AsyncStream<NetworkPathSnapshot> {
AsyncStream { continuation in
monitor.pathUpdateHandler = { path in
continuation.yield(Self.snapshot(for: path))
}
monitor.start(queue: queue)
continuation.onTermination = { [monitor] _ in
monitor.cancel()
}
}
}
private static func snapshot(for path: NWPath) -> NetworkPathSnapshot {
let link: NetworkPathSnapshot.LinkType
switch path.status {
case .satisfied:
if path.usesInterfaceType(.wifi) {
link = .wifi
} else if path.usesInterfaceType(.cellular) {
link = .cellular
} else if path.usesInterfaceType(.wiredEthernet) {
link = .wired
} else if path.usesInterfaceType(.loopback) {
link = .loopback
} else {
link = .other
}
default:
link = .unavailable
}
return NetworkPathSnapshot(
isAvailable: path.status == .satisfied,
isExpensive: path.isExpensive,
isConstrained: path.isConstrained,
link: link
)
}
}
public struct StaticPathProvider: NetworkPathProviding {
private let snapshot: NetworkPathSnapshot
public init(_ snapshot: NetworkPathSnapshot = .unknown) {
self.snapshot = snapshot
}
public var pathChanges: AsyncStream<NetworkPathSnapshot> {
let snapshot = self.snapshot
return AsyncStream { continuation in
continuation.yield(snapshot)
continuation.finish()
}
}
}

View File

@@ -0,0 +1,29 @@
import Foundation
public protocol PasswordProviding: Sendable {
func password(for keychainTag: String) -> String?
}
public struct DefaultPasswordProvider: PasswordProviding {
private let keychain: any KeychainServicing
public init(keychain: any KeychainServicing = KeychainService()) {
self.keychain = keychain
}
public func password(for keychainTag: String) -> String? {
try? keychain.loadPassword(account: keychainTag)
}
}
public struct StaticPasswordProvider: PasswordProviding {
private let password: String
public init(password: String) {
self.password = password
}
public func password(for keychainTag: String) -> String? {
password
}
}

View File

@@ -0,0 +1,14 @@
import Foundation
public enum PointerButton: Sendable, Hashable, CaseIterable {
case left
case middle
case right
}
public enum ScrollDirection: Sendable, Hashable, CaseIterable {
case up
case down
case left
case right
}

View File

@@ -0,0 +1,43 @@
import Foundation
public struct ReconnectPolicy: Sendable {
public let maxAttempts: Int
public let baseDelaySeconds: Double
public let maxDelaySeconds: Double
public let jitterFraction: Double
public init(
maxAttempts: Int = 6,
baseDelaySeconds: Double = 1.0,
maxDelaySeconds: Double = 30.0,
jitterFraction: Double = 0.25
) {
self.maxAttempts = maxAttempts
self.baseDelaySeconds = baseDelaySeconds
self.maxDelaySeconds = maxDelaySeconds
self.jitterFraction = jitterFraction
}
public static let `default` = ReconnectPolicy()
public static let none = ReconnectPolicy(maxAttempts: 0)
public func shouldReconnect(for reason: DisconnectReason) -> Bool {
guard maxAttempts > 0 else { return false }
switch reason {
case .userRequested, .authenticationFailed:
return false
case .networkError, .protocolError, .remoteClosed:
return true
}
}
public func delay(for attempt: Int) -> Double? {
guard attempt > 0, attempt <= maxAttempts else { return nil }
let exponential = baseDelaySeconds * pow(2.0, Double(attempt - 1))
let capped = min(exponential, maxDelaySeconds)
let jitter = capped * jitterFraction
let lower = max(0, capped - jitter)
let upper = capped + jitter
return Double.random(in: lower...upper)
}
}

View File

@@ -0,0 +1,12 @@
import Foundation
import CoreGraphics
public struct RemoteScreen: Identifiable, Hashable, Sendable {
public let id: UInt32
public let frame: CGRect
public init(id: UInt32, frame: CGRect) {
self.id = id
self.frame = frame
}
}

View File

@@ -0,0 +1,28 @@
import Foundation
#if canImport(UIKit)
import UIKit
#endif
struct SendableValueBox<Value>: @unchecked Sendable {
let value: Value
init(_ value: Value) { self.value = value }
}
enum ClipboardSink {
@MainActor
static func set(_ text: String) {
#if canImport(UIKit)
UIPasteboard.general.string = text
#endif
}
@MainActor
static func read() -> String? {
#if canImport(UIKit)
guard UIPasteboard.general.hasStrings else { return nil }
return UIPasteboard.general.string
#else
return nil
#endif
}
}

View File

@@ -1,42 +1,546 @@
import Foundation
import Observation
import CoreGraphics
import RoyalVNCKit
@Observable
@MainActor
@Observable
public final class SessionController {
public private(set) var state: SessionState = .idle
public private(set) var lastError: Error?
public private(set) var lastErrorMessage: String?
public private(set) var framebufferSize: FramebufferSize?
public private(set) var currentImage: CGImage?
public private(set) var imageRevision: Int = 0
public private(set) var lastUpdatedRegion: CGRect?
public private(set) var cursorImage: CGImage?
public private(set) var cursorHotspot: CGPoint = .zero
public private(set) var screens: [RemoteScreen] = []
public private(set) var desktopName: String?
public private(set) var isReconnecting: Bool = false
public private(set) var reconnectAttempt: Int = 0
private let transport: any Transport
private var runTask: Task<Void, Never>?
public var viewOnly: Bool
public var quality: QualityPreset {
didSet { applyQuality() }
}
public let clipboardSyncEnabled: Bool
public init(transport: any Transport) {
self.transport = transport
public let displayName: String
public let host: String
public let port: Int
public let username: String
private let keychainTag: String
private let passwordProvider: any PasswordProviding
private let preferredEncodings: [VNCFrameEncodingType]
private let pathProvider: any NetworkPathProviding
private let reconnectPolicy: ReconnectPolicy
private var connection: VNCConnection?
private var delegateAdapter: DelegateAdapter?
private var redrawScheduled = false
private var pendingDirtyRect: CGRect?
private var reconnectTask: Task<Void, Never>?
private var explicitlyStopped = false
private var pathObserver: Task<Void, Never>?
public init(
displayName: String,
host: String,
port: Int,
username: String = "",
keychainTag: String,
viewOnly: Bool = false,
clipboardSyncEnabled: Bool = true,
quality: QualityPreset = .adaptive,
preferredEncodings: [VNCFrameEncodingType] = .default,
passwordProvider: any PasswordProviding = DefaultPasswordProvider(),
pathProvider: any NetworkPathProviding = NWPathObserver(),
reconnectPolicy: ReconnectPolicy = .default
) {
self.displayName = displayName
self.host = host
self.port = port
self.username = username
self.keychainTag = keychainTag
self.viewOnly = viewOnly
self.clipboardSyncEnabled = clipboardSyncEnabled
self.quality = quality
self.preferredEncodings = preferredEncodings
self.passwordProvider = passwordProvider
self.pathProvider = pathProvider
self.reconnectPolicy = reconnectPolicy
}
public convenience init(
connection saved: SavedConnection,
passwordProvider: any PasswordProviding = DefaultPasswordProvider()
) {
let decoded = [VNCFrameEncodingType].decode(saved.preferredEncodings)
self.init(
displayName: saved.displayName,
host: saved.host,
port: saved.port,
username: saved.username,
keychainTag: saved.keychainTag,
viewOnly: saved.viewOnly,
clipboardSyncEnabled: saved.clipboardSyncEnabled,
quality: saved.quality,
preferredEncodings: decoded.isEmpty ? .default : decoded,
passwordProvider: passwordProvider
)
}
public func start() {
guard case .idle = state else { return }
state = .connecting
runTask = Task { [weak self] in
await self?.run()
explicitlyStopped = false
reconnectAttempt = 0
beginConnect()
startObservingPath()
}
public func stop() {
explicitlyStopped = true
reconnectTask?.cancel()
reconnectTask = nil
pathObserver?.cancel()
pathObserver = nil
connection?.disconnect()
}
public func reconnectNow() {
reconnectTask?.cancel()
reconnectTask = nil
connection?.disconnect()
beginConnect()
}
// MARK: Pointer
public func pointerMove(toNormalized point: CGPoint) {
guard let pos = framebufferPoint(for: point) else { return }
connection?.mouseMove(x: pos.x, y: pos.y)
}
public func pointerDown(_ button: PointerButton, atNormalized point: CGPoint) {
guard !viewOnly, let pos = framebufferPoint(for: point) else { return }
connection?.mouseButtonDown(button.vnc, x: pos.x, y: pos.y)
}
public func pointerUp(_ button: PointerButton, atNormalized point: CGPoint) {
guard !viewOnly, let pos = framebufferPoint(for: point) else { return }
connection?.mouseButtonUp(button.vnc, x: pos.x, y: pos.y)
}
public func pointerClick(_ button: PointerButton, atNormalized point: CGPoint) {
guard !viewOnly, let pos = framebufferPoint(for: point) else { return }
connection?.mouseMove(x: pos.x, y: pos.y)
connection?.mouseButtonDown(button.vnc, x: pos.x, y: pos.y)
connection?.mouseButtonUp(button.vnc, x: pos.x, y: pos.y)
}
public func pointerScroll(_ direction: ScrollDirection,
steps: UInt32,
atNormalized point: CGPoint) {
guard !viewOnly, steps > 0, let pos = framebufferPoint(for: point) else { return }
connection?.mouseWheel(direction.vnc, x: pos.x, y: pos.y, steps: steps)
}
// MARK: Keyboard
public func keyDown(keysym: UInt32) {
guard !viewOnly else { return }
connection?.keyDown(VNCKeyCode(keysym))
}
public func keyUp(keysym: UInt32) {
guard !viewOnly else { return }
connection?.keyUp(VNCKeyCode(keysym))
}
public func type(_ string: String) {
guard !viewOnly else { return }
let events = Self.keyEvents(for: string)
recordTypedEvents(events)
guard let connection else { return }
for event in events {
let code = VNCKeyCode(event.keysym)
if event.isDown {
connection.keyDown(code)
} else {
connection.keyUp(code)
}
}
}
public func stop() async {
runTask?.cancel()
await transport.disconnect()
state = .disconnected(reason: .userRequested)
/// Pure mapping from a string to the key down/up keysym pairs it should
/// generate. Newlines map to X11 Return; all other printable ASCII maps
/// 1:1 to the equivalent X11 keysym (which equals the ASCII code).
public nonisolated static func keyEvents(for string: String) -> [KeyEvent] {
var events: [KeyEvent] = []
for char in string {
if char.isNewline {
events.append(KeyEvent(keysym: 0xFF0D, isDown: true))
events.append(KeyEvent(keysym: 0xFF0D, isDown: false))
continue
}
for code in VNCKeyCode.withCharacter(char) {
events.append(KeyEvent(keysym: code.rawValue, isDown: true))
events.append(KeyEvent(keysym: code.rawValue, isDown: false))
}
}
return events
}
private func run() async {
do {
try await transport.connect()
state = .authenticating
// Phase 1 will plug RoyalVNCKit.VNCConnection here and drive its
// state machine from the transport byte stream.
} catch {
lastError = error
state = .disconnected(reason: .networkError(String(describing: error)))
public struct KeyEvent: Equatable, Sendable {
public let keysym: UInt32
public let isDown: Bool
}
/// Test hook: when set, every typed event is appended here in addition to
/// being sent over the wire. Used by UI tests to verify keyboard plumbing.
public var typedEventLog: [KeyEvent] = []
public var typedEventLogEnabled: Bool = false
private func recordTypedEvents(_ events: [KeyEvent]) {
guard typedEventLogEnabled else { return }
typedEventLog.append(contentsOf: events)
}
public func sendBackspace() { pressKey(.delete) }
public func sendReturn() { pressKey(.return) }
public func sendEscape() { pressKey(.escape) }
public func sendTab() { pressKey(.tab) }
public func sendArrow(_ direction: ScrollDirection) {
switch direction {
case .up: pressKey(.upArrow)
case .down: pressKey(.downArrow)
case .left: pressKey(.leftArrow)
case .right: pressKey(.rightArrow)
}
}
public func sendFunctionKey(_ index: Int) {
guard let code = VNCKeyCode.functionKey(index) else { return }
pressKey(code)
}
public func pressKeyCombo(_ keys: [VNCKeyCode]) {
guard !viewOnly, let connection else { return }
for code in keys { connection.keyDown(code) }
for code in keys.reversed() { connection.keyUp(code) }
}
private func pressKey(_ code: VNCKeyCode) {
guard !viewOnly else { return }
connection?.keyDown(code)
connection?.keyUp(code)
}
// MARK: Clipboard
public func pushLocalClipboardNow(_ text: String) {
ClipboardSink.set(text)
}
// MARK: Internal handlers (invoked on MainActor by adapter)
func handleConnectionState(status: VNCConnection.Status, errorMessage: String?) {
switch status {
case .connecting:
state = .connecting
case .connected:
isReconnecting = false
reconnectAttempt = 0
if let size = framebufferSize {
state = .connected(framebufferSize: size)
} else {
state = .authenticating
}
case .disconnecting:
break
case .disconnected:
let reason: DisconnectReason
if let errorMessage {
if errorMessage.lowercased().contains("authentication") {
reason = .authenticationFailed
} else {
reason = .networkError(errorMessage)
}
lastErrorMessage = errorMessage
} else {
reason = explicitlyStopped ? .userRequested : .remoteClosed
}
state = .disconnected(reason: reason)
connection?.delegate = nil
connection = nil
delegateAdapter = nil
if !explicitlyStopped, reconnectPolicy.shouldReconnect(for: reason) {
scheduleReconnect()
}
}
}
func handleFramebufferCreated(size: FramebufferSize, screens: [RemoteScreen]) {
framebufferSize = size
self.screens = screens
state = .connected(framebufferSize: size)
applyQuality()
}
func handleFramebufferResized(size: FramebufferSize, screens: [RemoteScreen]) {
framebufferSize = size
self.screens = screens
state = .connected(framebufferSize: size)
}
func handleFramebufferUpdated(region: CGRect) {
if let existing = pendingDirtyRect {
pendingDirtyRect = existing.union(region)
} else {
pendingDirtyRect = region
}
guard !redrawScheduled else { return }
redrawScheduled = true
Task { @MainActor in
self.redrawScheduled = false
self.refreshImage()
}
}
func handleCursorUpdate(image: CGImage?, hotspot: CGPoint) {
cursorImage = image
cursorHotspot = hotspot
}
func refreshImage() {
guard let connection, let fb = connection.framebuffer else { return }
currentImage = fb.cgImage
lastUpdatedRegion = pendingDirtyRect
pendingDirtyRect = nil
imageRevision &+= 1
}
// MARK: Helpers
private func beginConnect() {
if connection != nil { return }
state = .connecting
lastErrorMessage = nil
let settings = VNCConnection.Settings(
isDebugLoggingEnabled: false,
hostname: host,
port: UInt16(clamping: port),
isShared: true,
isScalingEnabled: false,
useDisplayLink: false,
// Any value other than .none, otherwise RoyalVNCKit drops every
// enqueued PointerEvent/KeyEvent. On iOS we don't have local
// keyboard shortcuts to worry about, so pick the most permissive
// forwarding mode.
inputMode: .forwardKeyboardShortcutsEvenIfInUseLocally,
isClipboardRedirectionEnabled: clipboardSyncEnabled,
colorDepth: .depth24Bit,
frameEncodings: preferredEncodings
)
let connection = VNCConnection(settings: settings)
let adapter = DelegateAdapter(
controller: self,
passwordProvider: passwordProvider,
keychainTag: keychainTag,
username: username
)
connection.delegate = adapter
self.connection = connection
self.delegateAdapter = adapter
connection.connect()
}
private func scheduleReconnect() {
let attempt = reconnectAttempt + 1
reconnectAttempt = attempt
guard let delay = reconnectPolicy.delay(for: attempt) else {
isReconnecting = false
return
}
isReconnecting = true
reconnectTask?.cancel()
reconnectTask = Task { @MainActor [weak self] in
try? await Task.sleep(for: .milliseconds(Int(delay * 1000)))
guard let self, !self.explicitlyStopped else { return }
self.beginConnect()
}
}
private func startObservingPath() {
pathObserver?.cancel()
let stream = pathProvider.pathChanges
pathObserver = Task { @MainActor [weak self] in
for await change in stream {
guard let self else { return }
if change.isExpensive != self.lastPathExpensive {
self.lastPathExpensive = change.isExpensive
self.applyQuality()
}
}
}
}
private var lastPathExpensive: Bool = false
private func applyQuality() {
// Phase 2 hook: with extended encodings we'd push CompressionLevel/JPEGQuality
// pseudo-encodings here. RoyalVNCKit currently picks fixed levels internally;
// the connection re-orders frame encodings on next handshake. For now this is
// a no-op stub; the field is wired so QualityPreset round-trips through state.
_ = quality
}
private func framebufferPoint(for normalized: CGPoint) -> (x: UInt16, y: UInt16)? {
guard let size = framebufferSize else { return nil }
let nx = max(0, min(1, normalized.x))
let ny = max(0, min(1, normalized.y))
let x = UInt16(clamping: Int((nx * CGFloat(size.width - 1)).rounded()))
let y = UInt16(clamping: Int((ny * CGFloat(size.height - 1)).rounded()))
return (x, y)
}
}
// MARK: - Delegate Adapter
private final class DelegateAdapter: NSObject, VNCConnectionDelegate, @unchecked Sendable {
weak var controller: SessionController?
let passwordProvider: any PasswordProviding
let keychainTag: String
let username: String
init(controller: SessionController,
passwordProvider: any PasswordProviding,
keychainTag: String,
username: String) {
self.controller = controller
self.passwordProvider = passwordProvider
self.keychainTag = keychainTag
self.username = username
}
func connection(_ connection: VNCConnection,
stateDidChange newState: VNCConnection.ConnectionState) {
let status = newState.status
let errorMessage: String? = newState.error.flatMap { error in
if let localized = error as? LocalizedError, let desc = localized.errorDescription {
return desc
}
return String(describing: error)
}
Task { @MainActor in
self.controller?.handleConnectionState(status: status, errorMessage: errorMessage)
}
}
func connection(_ connection: VNCConnection,
credentialFor authenticationType: VNCAuthenticationType,
completion: @escaping (VNCCredential?) -> Void) {
let pwd = passwordProvider.password(for: keychainTag) ?? ""
let credential: VNCCredential
switch authenticationType {
case .vnc:
credential = VNCPasswordCredential(password: pwd)
case .appleRemoteDesktop, .ultraVNCMSLogonII:
credential = VNCUsernamePasswordCredential(username: username, password: pwd)
@unknown default:
credential = VNCPasswordCredential(password: pwd)
}
completion(credential)
}
func connection(_ connection: VNCConnection,
didCreateFramebuffer framebuffer: VNCFramebuffer) {
let size = FramebufferSize(width: Int(framebuffer.size.width),
height: Int(framebuffer.size.height))
let screens = framebuffer.screens.map {
RemoteScreen(id: $0.id, frame: $0.cgFrame)
}
Task { @MainActor in
self.controller?.handleFramebufferCreated(size: size, screens: screens)
self.controller?.refreshImage()
}
}
func connection(_ connection: VNCConnection,
didResizeFramebuffer framebuffer: VNCFramebuffer) {
let size = FramebufferSize(width: Int(framebuffer.size.width),
height: Int(framebuffer.size.height))
let screens = framebuffer.screens.map {
RemoteScreen(id: $0.id, frame: $0.cgFrame)
}
Task { @MainActor in
self.controller?.handleFramebufferResized(size: size, screens: screens)
self.controller?.refreshImage()
}
}
func connection(_ connection: VNCConnection,
didUpdateFramebuffer framebuffer: VNCFramebuffer,
x: UInt16, y: UInt16,
width: UInt16, height: UInt16) {
let region = CGRect(x: Int(x), y: Int(y), width: Int(width), height: Int(height))
Task { @MainActor in
self.controller?.handleFramebufferUpdated(region: region)
}
}
func connection(_ connection: VNCConnection,
didUpdateCursor cursor: VNCCursor) {
let imageBox = SendableValueBox(cursor.cgImage)
let hotspot = cursor.cgHotspot
Task { @MainActor in
self.controller?.handleCursorUpdate(image: imageBox.value, hotspot: hotspot)
}
}
}
private extension VNCKeyCode {
static func functionKey(_ index: Int) -> VNCKeyCode? {
switch index {
case 1: .f1
case 2: .f2
case 3: .f3
case 4: .f4
case 5: .f5
case 6: .f6
case 7: .f7
case 8: .f8
case 9: .f9
case 10: .f10
case 11: .f11
case 12: .f12
case 13: .f13
case 14: .f14
case 15: .f15
case 16: .f16
case 17: .f17
case 18: .f18
case 19: .f19
default: nil
}
}
}
private extension PointerButton {
var vnc: VNCMouseButton {
switch self {
case .left: .left
case .middle: .middle
case .right: .right
}
}
}
private extension ScrollDirection {
var vnc: VNCMouseWheel {
switch self {
case .up: .up
case .down: .down
case .left: .left
case .right: .right
}
}
}

View File

@@ -7,37 +7,49 @@ public final class SavedConnection {
public var displayName: String
public var host: String
public var port: Int
public var username: String
public var colorTagRaw: String
public var lastConnectedAt: Date?
public var preferredEncodings: [String]
public var keychainTag: String
public var qualityRaw: String
public var inputModeRaw: String
public var viewOnly: Bool
public var curtainMode: Bool
public var clipboardSyncEnabled: Bool
public var notes: String
public init(
id: UUID = UUID(),
displayName: String,
host: String,
port: Int = 5900,
username: String = "",
colorTag: ColorTag = .blue,
preferredEncodings: [String] = ["tight", "zrle", "hextile", "raw"],
preferredEncodings: [String] = ["7", "16", "5", "6"],
keychainTag: String = UUID().uuidString,
quality: QualityPreset = .adaptive,
inputMode: InputModePreference = .touch,
viewOnly: Bool = false,
curtainMode: Bool = false
curtainMode: Bool = false,
clipboardSyncEnabled: Bool = true,
notes: String = ""
) {
self.id = id
self.displayName = displayName
self.host = host
self.port = port
self.username = username
self.colorTagRaw = colorTag.rawValue
self.lastConnectedAt = nil
self.preferredEncodings = preferredEncodings
self.keychainTag = keychainTag
self.qualityRaw = quality.rawValue
self.inputModeRaw = inputMode.rawValue
self.viewOnly = viewOnly
self.curtainMode = curtainMode
self.clipboardSyncEnabled = clipboardSyncEnabled
self.notes = notes
}
public var colorTag: ColorTag {
@@ -49,6 +61,11 @@ public final class SavedConnection {
get { QualityPreset(rawValue: qualityRaw) ?? .adaptive }
set { qualityRaw = newValue.rawValue }
}
public var inputMode: InputModePreference {
get { InputModePreference(rawValue: inputModeRaw) ?? .touch }
set { inputModeRaw = newValue.rawValue }
}
}
public enum ColorTag: String, CaseIterable, Sendable {
@@ -58,3 +75,8 @@ public enum ColorTag: String, CaseIterable, Sendable {
public enum QualityPreset: String, CaseIterable, Sendable {
case adaptive, high, low
}
public enum InputModePreference: String, CaseIterable, Sendable {
case touch
case trackpad
}

View File

@@ -0,0 +1,41 @@
import Testing
@testable import VNCCore
import Foundation
@Suite struct KeyboardInputTests {
@Test func plainAsciiProducesDownUpPairs() {
let events = SessionController.keyEvents(for: "ab")
#expect(events.count == 4)
#expect(events[0] == .init(keysym: 0x61, isDown: true))
#expect(events[1] == .init(keysym: 0x61, isDown: false))
#expect(events[2] == .init(keysym: 0x62, isDown: true))
#expect(events[3] == .init(keysym: 0x62, isDown: false))
}
@Test func uppercaseAsciiSendsDistinctKeysym() {
let events = SessionController.keyEvents(for: "A")
#expect(events.count == 2)
#expect(events[0] == .init(keysym: 0x41, isDown: true))
#expect(events[1] == .init(keysym: 0x41, isDown: false))
}
@Test func spaceMapsToAsciiSpace() {
let events = SessionController.keyEvents(for: " ")
#expect(events.count == 2)
#expect(events[0].keysym == 0x20)
}
@Test func newlineMapsToX11Return() {
let events = SessionController.keyEvents(for: "\n")
#expect(events.count == 2)
#expect(events[0] == .init(keysym: 0xFF0D, isDown: true))
#expect(events[1] == .init(keysym: 0xFF0D, isDown: false))
}
@Test func passwordWithMixedCaseAndDigitsAndPunctuation() {
let events = SessionController.keyEvents(for: "Hi!7")
let downKeys = events.filter(\.isDown).map(\.keysym)
// H=0x48, i=0x69, !=0x21, 7=0x37
#expect(downKeys == [0x48, 0x69, 0x21, 0x37])
}
}

View File

@@ -1,6 +1,7 @@
import Testing
@testable import VNCCore
import Foundation
import CoreGraphics
@Suite struct SessionStateTests {
@Test func idleEqualsIdle() {
@@ -17,3 +18,74 @@ import Foundation
#expect(DisconnectReason.userRequested != .authenticationFailed)
}
}
@Suite struct ReconnectPolicyTests {
@Test func userRequestedNeverReconnects() {
let policy = ReconnectPolicy.default
#expect(!policy.shouldReconnect(for: .userRequested))
#expect(!policy.shouldReconnect(for: .authenticationFailed))
}
@Test func networkFailuresReconnect() {
let policy = ReconnectPolicy.default
#expect(policy.shouldReconnect(for: .networkError("oops")))
#expect(policy.shouldReconnect(for: .remoteClosed))
}
@Test func policyDelaysGrowAndCap() {
let policy = ReconnectPolicy(maxAttempts: 5,
baseDelaySeconds: 1,
maxDelaySeconds: 8,
jitterFraction: 0)
#expect(policy.delay(for: 1) == 1)
#expect(policy.delay(for: 2) == 2)
#expect(policy.delay(for: 3) == 4)
#expect(policy.delay(for: 4) == 8)
#expect(policy.delay(for: 5) == 8)
#expect(policy.delay(for: 6) == nil)
}
@Test func zeroAttemptsDisablesReconnect() {
let policy = ReconnectPolicy.none
#expect(!policy.shouldReconnect(for: .networkError("x")))
}
}
@Suite struct RemoteScreenTests {
@Test func screensAreHashable() {
let a = RemoteScreen(id: 1, frame: CGRect(x: 0, y: 0, width: 1920, height: 1080))
let b = RemoteScreen(id: 1, frame: CGRect(x: 0, y: 0, width: 1920, height: 1080))
let c = RemoteScreen(id: 2, frame: CGRect(x: 0, y: 0, width: 1920, height: 1080))
#expect(a == b)
#expect(a != c)
#expect(Set([a, b, c]).count == 2)
}
}
@Suite struct PasswordProviderTests {
private final class StubKeychain: KeychainServicing, @unchecked Sendable {
var stored: [String: String] = [:]
func storePassword(_ password: String, account: String) throws {
stored[account] = password
}
func loadPassword(account: String) throws -> String? {
stored[account]
}
func deletePassword(account: String) throws {
stored[account] = nil
}
}
@Test func keychainBackedProviderReturnsStored() {
let keychain = StubKeychain()
try? keychain.storePassword("hunter2", account: "abc")
let provider = DefaultPasswordProvider(keychain: keychain)
#expect(provider.password(for: "abc") == "hunter2")
#expect(provider.password(for: "missing") == nil)
}
@Test func staticProviderAlwaysReturnsSame() {
let provider = StaticPasswordProvider(password: "fixed")
#expect(provider.password(for: "anything") == "fixed")
}
}

View File

@@ -2,45 +2,156 @@ import SwiftUI
import SwiftData
import VNCCore
public struct AddConnectionPrefill: Equatable, Sendable {
public let displayName: String
public let host: String
public let port: Int
public init(displayName: String = "", host: String = "", port: Int = 5900) {
self.displayName = displayName
self.host = host
self.port = port
}
}
public struct AddConnectionView: View {
@Environment(\.modelContext) private var context
@Environment(\.dismiss) private var dismiss
let prefill: AddConnectionPrefill?
let editing: SavedConnection?
@State private var displayName = ""
@State private var host = ""
@State private var port = "5900"
@State private var username = ""
@State private var password = ""
@State private var revealPassword = false
@State private var colorTag: ColorTag = .blue
@State private var quality: QualityPreset = .adaptive
@State private var inputMode: InputModePreference = .touch
@State private var clipboardSync = true
@State private var viewOnly = false
@State private var notes = ""
@State private var hasLoadedExisting = false
public init() {}
public init(prefill: AddConnectionPrefill? = nil, editing: SavedConnection? = nil) {
self.prefill = prefill
self.editing = editing
}
private var isEditing: Bool { editing != nil }
private var canSave: Bool {
!displayName.trimmingCharacters(in: .whitespaces).isEmpty &&
!host.trimmingCharacters(in: .whitespaces).isEmpty
}
private var authFooterText: String {
if isEditing {
return "Leave password blank to keep the current one. Stored in iOS Keychain (this device only)."
}
return """
Two Mac paths:
• User account (ARD) — enter your macOS short name + full account password. No length limit. Enabled in System Settings → Sharing → Screen Sharing → Allow access for…
• VNC-only password — leave the username blank; macOS truncates to 8 characters. Enabled via Screen Sharing → ⓘ → "VNC viewers may control screen with password".
"""
}
public var body: some View {
NavigationStack {
Form {
Section("Connection") {
Section {
TextField("Display name", text: $displayName)
.font(.headline)
TextField("Host or IP", text: $host)
#if os(iOS)
.textInputAutocapitalization(.never)
#endif
.autocorrectionDisabled()
TextField("Port", text: $port)
.font(.body.monospacedDigit())
HStack {
Text("Port")
Spacer()
TextField("5900", text: $port)
#if os(iOS)
.keyboardType(.numberPad)
#endif
.multilineTextAlignment(.trailing)
.font(.body.monospacedDigit())
.frame(maxWidth: 100)
}
} header: {
Text("Connection")
} footer: {
Text("Tailscale IPs and MagicDNS names work as long as the Tailscale app is connected.")
}
Section {
TextField("Username (optional)", text: $username)
#if os(iOS)
.keyboardType(.numberPad)
.textInputAutocapitalization(.never)
#endif
.autocorrectionDisabled()
HStack {
Group {
if revealPassword {
TextField("Password", text: $password)
} else {
SecureField(isEditing ? "Replace password" : "Password",
text: $password)
}
}
.font(.body)
Button {
revealPassword.toggle()
} label: {
Image(systemName: revealPassword ? "eye.slash" : "eye")
.foregroundStyle(.secondary)
}
.buttonStyle(.plain)
}
} header: {
Text("Authentication")
} footer: {
Text(authFooterText)
}
Section("Authentication") {
SecureField("VNC password", text: $password)
Section {
colorTagPicker
.listRowInsets(EdgeInsets(top: 4, leading: 16, bottom: 8, trailing: 16))
} header: {
Text("Tag")
}
Section("Appearance") {
Picker("Color tag", selection: $colorTag) {
ForEach(ColorTag.allCases, id: \.self) { tag in
Text(tag.rawValue.capitalized).tag(tag)
Section {
Picker("Default input", selection: $inputMode) {
ForEach(InputModePreference.allCases, id: \.self) { mode in
Text(mode == .touch ? "Touch" : "Trackpad").tag(mode)
}
}
Picker("Quality", selection: $quality) {
Text("Adaptive").tag(QualityPreset.adaptive)
Text("High").tag(QualityPreset.high)
Text("Low (slow links)").tag(QualityPreset.low)
}
Toggle("Sync clipboard", isOn: $clipboardSync)
Toggle("View only", isOn: $viewOnly)
} header: {
Text("Defaults")
} footer: {
Text("View-only sends no input to the remote computer — useful as a watchdog screen.")
}
Section("Notes") {
TextField("Optional", text: $notes, axis: .vertical)
.lineLimit(2...6)
}
}
.navigationTitle("New Connection")
#if os(iOS)
.scrollContentBackground(.hidden)
#endif
.background(formBackground.ignoresSafeArea())
.navigationTitle(isEditing ? "Edit Connection" : "New Connection")
#if os(iOS)
.navigationBarTitleDisplayMode(.inline)
#endif
@@ -49,24 +160,109 @@ public struct AddConnectionView: View {
Button("Cancel") { dismiss() }
}
ToolbarItem(placement: .confirmationAction) {
Button("Save") { save() }
.disabled(displayName.isEmpty || host.isEmpty)
Button(isEditing ? "Save" : "Add") { save() }
.disabled(!canSave)
}
}
.onAppear { loadExistingIfNeeded() }
}
}
private var formBackground: some View {
LinearGradient(
colors: [
Color(red: 0.04, green: 0.05, blue: 0.10),
Color(red: 0.02, green: 0.02, blue: 0.05)
],
startPoint: .top,
endPoint: .bottom
)
}
private var colorTagPicker: some View {
ScrollView(.horizontal, showsIndicators: false) {
HStack(spacing: 12) {
ForEach(ColorTag.allCases, id: \.self) { tag in
Button {
colorTag = tag
} label: {
ZStack {
Circle()
.fill(tag.color)
.frame(width: 28, height: 28)
if tag == colorTag {
Circle()
.stroke(Color.primary, lineWidth: 2)
.frame(width: 36, height: 36)
}
}
.frame(width: 40, height: 40)
.accessibilityLabel(tag.rawValue.capitalized)
.accessibilityAddTraits(tag == colorTag ? .isSelected : [])
}
.buttonStyle(.plain)
}
}
.padding(.vertical, 4)
}
}
private func loadExistingIfNeeded() {
guard !hasLoadedExisting else { return }
hasLoadedExisting = true
if let existing = editing {
displayName = existing.displayName
host = existing.host
port = String(existing.port)
username = existing.username
colorTag = existing.colorTag
quality = existing.quality
inputMode = existing.inputMode
clipboardSync = existing.clipboardSyncEnabled
viewOnly = existing.viewOnly
notes = existing.notes
} else if let prefill, displayName.isEmpty, host.isEmpty {
displayName = prefill.displayName
host = prefill.host
port = String(prefill.port)
}
}
private func save() {
let portInt = Int(port) ?? 5900
let connection = SavedConnection(
displayName: displayName,
host: host,
port: portInt,
colorTag: colorTag
)
context.insert(connection)
if !password.isEmpty {
try? KeychainService().storePassword(password, account: connection.keychainTag)
let trimmedUsername = username.trimmingCharacters(in: .whitespaces)
if let existing = editing {
existing.displayName = displayName
existing.host = host
existing.port = portInt
existing.username = trimmedUsername
existing.colorTag = colorTag
existing.quality = quality
existing.inputMode = inputMode
existing.clipboardSyncEnabled = clipboardSync
existing.viewOnly = viewOnly
existing.notes = notes
if !password.isEmpty {
try? KeychainService().storePassword(password, account: existing.keychainTag)
}
} else {
let connection = SavedConnection(
displayName: displayName,
host: host,
port: portInt,
username: trimmedUsername,
colorTag: colorTag,
quality: quality,
inputMode: inputMode,
viewOnly: viewOnly,
curtainMode: false,
clipboardSyncEnabled: clipboardSync,
notes: notes
)
context.insert(connection)
if !password.isEmpty {
try? KeychainService().storePassword(password, account: connection.keychainTag)
}
}
try? context.save()
dismiss()

View File

@@ -5,29 +5,51 @@ struct ConnectionCard: View {
let connection: SavedConnection
var body: some View {
HStack(spacing: 12) {
Circle()
.fill(connection.colorTag.color)
.frame(width: 12, height: 12)
HStack(spacing: 14) {
ZStack {
Circle()
.fill(connection.colorTag.color.opacity(0.18))
.frame(width: 38, height: 38)
Circle()
.fill(connection.colorTag.color)
.frame(width: 14, height: 14)
}
VStack(alignment: .leading, spacing: 2) {
Text(connection.displayName)
.font(.headline)
Text("\(connection.host):\(connection.port)")
.font(.caption)
.foregroundStyle(.primary)
.lineLimit(1)
Text("\(connection.host):\(portLabel(connection.port))")
.font(.caption.monospacedDigit())
.foregroundStyle(.secondary)
.lineLimit(1)
.truncationMode(.middle)
if connection.viewOnly {
Label("View only", systemImage: "eye")
.labelStyle(.titleAndIcon)
.font(.caption2)
.foregroundStyle(.tertiary)
}
}
Spacer()
if let last = connection.lastConnectedAt {
Text(last, style: .relative)
.font(.caption2)
Spacer(minLength: 8)
VStack(alignment: .trailing, spacing: 4) {
if let last = connection.lastConnectedAt {
Text(last, format: .relative(presentation: .numeric, unitsStyle: .abbreviated))
.font(.caption2)
.foregroundStyle(.tertiary)
.lineLimit(1)
}
Image(systemName: "chevron.right")
.font(.caption2.weight(.bold))
.foregroundStyle(.tertiary)
}
}
.padding(.vertical, 6)
.contentShape(Rectangle())
}
}
private extension ColorTag {
extension ColorTag {
var color: Color {
switch self {
case .red: .red

View File

@@ -4,66 +4,388 @@ import VNCCore
public struct ConnectionListView: View {
@Environment(\.modelContext) private var modelContext
@Environment(\.openWindow) private var openWindow
@Query(sort: \SavedConnection.displayName) private var connections: [SavedConnection]
@State private var discovery = DiscoveryService()
@State private var showingAdd = false
@State private var selectedConnection: SavedConnection?
@State private var showingSettings = false
@State private var addPrefill: AddConnectionPrefill?
@State private var editingConnection: SavedConnection?
@State private var sessionConnection: SavedConnection?
@State private var resolvingHostID: String?
@State private var search = ""
public init() {}
public var body: some View {
NavigationStack {
List {
if !discovery.hosts.isEmpty {
Section("Discovered on this network") {
ForEach(discovery.hosts) { host in
Button {
// Phase 1: resolve host to SavedConnection draft
} label: {
Label(host.displayName, systemImage: "bonjour")
}
}
}
VStack(spacing: 0) {
topChrome
contentList
}
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .top)
.background(backgroundGradient.ignoresSafeArea())
.sheet(isPresented: $showingAdd) {
AddConnectionView(prefill: addPrefill)
}
.sheet(item: $editingConnection) { connection in
AddConnectionView(editing: connection)
}
.sheet(isPresented: $showingSettings) {
SettingsView()
}
#if os(iOS)
.fullScreenCover(item: $sessionConnection) { connection in
SessionView(connection: connection)
}
#else
.sheet(item: $sessionConnection) { connection in
SessionView(connection: connection)
}
#endif
.task { discovery.start() }
.onDisappear { discovery.stop() }
}
// MARK: Background
private var backgroundGradient: some View {
ZStack {
LinearGradient(
colors: [
Color(red: 0.06, green: 0.08, blue: 0.16),
Color(red: 0.02, green: 0.02, blue: 0.06)
],
startPoint: .top,
endPoint: .bottom
)
RadialGradient(
colors: [Color.blue.opacity(0.20), .clear],
center: .init(x: 0.5, y: 0.05),
startRadius: 10,
endRadius: 360
)
.blendMode(.screen)
.allowsHitTesting(false)
}
}
// MARK: Top chrome flush with safe area top
private var topChrome: some View {
VStack(spacing: 12) {
HStack(alignment: .center) {
circularGlassButton(systemName: "gearshape", label: "Settings") {
showingSettings = true
}
Section("Saved") {
if connections.isEmpty {
ContentUnavailableView(
"No saved connections",
systemImage: "display",
description: Text("Tap + to add a computer to connect to.")
)
} else {
ForEach(connections) { connection in
ConnectionCard(connection: connection)
.onTapGesture {
selectedConnection = connection
Spacer()
Text("Screens")
.font(.system(size: 26, weight: .bold, design: .rounded))
.foregroundStyle(.white)
Spacer()
circularGlassButton(systemName: "plus", label: "Add connection") {
addPrefill = nil
showingAdd = true
}
}
searchField
}
.padding(.horizontal, 16)
.padding(.top, 8)
.padding(.bottom, 12)
}
private func circularGlassButton(systemName: String,
label: String,
action: @escaping () -> Void) -> some View {
Button(action: action) {
Image(systemName: systemName)
.font(.callout.weight(.semibold))
.foregroundStyle(.white)
.frame(width: 38, height: 38)
}
.buttonStyle(.plain)
.glassSurface(in: Circle())
.accessibilityLabel(label)
}
private var searchField: some View {
HStack(spacing: 10) {
Image(systemName: "magnifyingglass")
.foregroundStyle(.white.opacity(0.65))
TextField("", text: $search, prompt: Text("Search connections")
.foregroundStyle(.white.opacity(0.45)))
.textFieldStyle(.plain)
.foregroundStyle(.white)
#if os(iOS)
.textInputAutocapitalization(.never)
#endif
.autocorrectionDisabled()
.submitLabel(.search)
if !search.isEmpty {
Button {
search = ""
} label: {
Image(systemName: "xmark.circle.fill")
.foregroundStyle(.white.opacity(0.45))
}
.buttonStyle(.plain)
.accessibilityLabel("Clear search")
}
}
.padding(.horizontal, 14)
.padding(.vertical, 11)
.glassSurface(in: Capsule())
}
// MARK: Content
private var contentList: some View {
ScrollView {
LazyVStack(spacing: 22) {
if !discovery.hosts.isEmpty || discovery.isBrowsing {
discoveredSection
}
savedSection
Color.clear.frame(height: 1) // bottom anchor
}
.padding(.horizontal, 16)
.padding(.top, 4)
.padding(.bottom, 24)
}
#if os(iOS)
.scrollIndicators(.hidden)
#endif
}
// MARK: Discovered
private var discoveredSection: some View {
VStack(alignment: .leading, spacing: 10) {
sectionHeader("On this network",
systemImage: "antenna.radiowaves.left.and.right")
VStack(spacing: 10) {
if discovery.hosts.isEmpty {
HStack(spacing: 12) {
ProgressView().controlSize(.small).tint(.white)
Text("Looking for computers…")
.font(.callout)
.foregroundStyle(.white.opacity(0.7))
Spacer()
}
.padding(16)
.glassSurface(in: RoundedRectangle(cornerRadius: 18, style: .continuous))
} else {
ForEach(discovery.hosts) { host in
Button {
Task { await prepareDiscoveredHost(host) }
} label: {
HStack(spacing: 14) {
ZStack {
Circle()
.fill(Color.blue.opacity(0.25))
.frame(width: 38, height: 38)
Image(systemName: "wifi")
.font(.callout)
.foregroundStyle(.blue)
}
VStack(alignment: .leading, spacing: 2) {
Text(host.displayName)
.font(.headline)
.foregroundStyle(.white)
Text(serviceLabel(host.serviceType))
.font(.caption)
.foregroundStyle(.white.opacity(0.6))
}
Spacer()
if resolvingHostID == host.id {
ProgressView().controlSize(.small).tint(.white)
} else {
Image(systemName: "plus.circle.fill")
.font(.title3)
.foregroundStyle(.blue)
}
}
.padding(14)
}
.buttonStyle(.plain)
.glassSurface(in: RoundedRectangle(cornerRadius: 18, style: .continuous))
.disabled(resolvingHostID != nil)
}
}
}
.navigationTitle("Screens")
.toolbar {
ToolbarItem(placement: .primaryAction) {
Button {
showingAdd = true
} label: {
Image(systemName: "plus")
}
}
}
.sheet(isPresented: $showingAdd) {
AddConnectionView()
}
.navigationDestination(item: $selectedConnection) { connection in
SessionView(connection: connection)
}
.task {
discovery.start()
}
.onDisappear {
discovery.stop()
}
}
}
// MARK: Saved
private var savedSection: some View {
VStack(alignment: .leading, spacing: 10) {
sectionHeader("Saved", systemImage: "bookmark.fill")
if filteredConnections.isEmpty {
if connections.isEmpty {
emptySavedState
} else {
noSearchResultsState
}
} else {
VStack(spacing: 10) {
ForEach(filteredConnections) { connection in
connectionRow(connection)
}
}
}
}
}
private var emptySavedState: some View {
VStack(spacing: 14) {
ZStack {
Circle()
.fill(LinearGradient(colors: [.blue.opacity(0.4), .purple.opacity(0.3)],
startPoint: .topLeading,
endPoint: .bottomTrailing))
.frame(width: 64, height: 64)
Image(systemName: "display")
.font(.title.weight(.semibold))
.foregroundStyle(.white)
}
Text("No saved connections")
.font(.headline)
.foregroundStyle(.white)
Text("Tap + at the top to add a Mac, Linux box, or Raspberry Pi.")
.font(.callout)
.foregroundStyle(.white.opacity(0.65))
.multilineTextAlignment(.center)
.padding(.horizontal, 8)
Button {
addPrefill = nil
showingAdd = true
} label: {
Label("Add a computer", systemImage: "plus")
.font(.callout.weight(.semibold))
.padding(.horizontal, 18)
.padding(.vertical, 12)
}
.buttonStyle(.plain)
.background(
Capsule().fill(LinearGradient(colors: [.blue, .purple],
startPoint: .leading,
endPoint: .trailing))
)
.foregroundStyle(.white)
}
.frame(maxWidth: .infinity)
.padding(28)
.glassSurface(in: RoundedRectangle(cornerRadius: 22, style: .continuous))
}
private var noSearchResultsState: some View {
VStack(spacing: 8) {
Image(systemName: "magnifyingglass")
.font(.title)
.foregroundStyle(.white.opacity(0.5))
Text("No matches for \"\(search)\"")
.font(.callout)
.foregroundStyle(.white.opacity(0.65))
}
.frame(maxWidth: .infinity)
.padding(24)
.glassSurface(in: RoundedRectangle(cornerRadius: 18, style: .continuous))
}
private func connectionRow(_ connection: SavedConnection) -> some View {
Button {
sessionConnection = connection
} label: {
ConnectionCard(connection: connection)
.padding(14)
}
.buttonStyle(.plain)
.glassSurface(in: RoundedRectangle(cornerRadius: 18, style: .continuous))
.contextMenu {
Button {
editingConnection = connection
} label: {
Label("Edit", systemImage: "pencil")
}
#if os(iOS)
Button {
openWindow(value: connection.id)
} label: {
Label("Open in New Window", systemImage: "rectangle.stack.badge.plus")
}
#endif
Divider()
Button(role: .destructive) {
delete(connection)
} label: {
Label("Delete", systemImage: "trash")
}
}
}
// MARK: Helpers
private func sectionHeader(_ title: String, systemImage: String) -> some View {
HStack(spacing: 8) {
Image(systemName: systemImage)
.font(.caption.weight(.semibold))
Text(title)
.font(.subheadline.weight(.semibold))
.textCase(.uppercase)
.tracking(0.5)
}
.foregroundStyle(.white.opacity(0.55))
.padding(.horizontal, 4)
}
private var filteredConnections: [SavedConnection] {
let trimmed = search.trimmingCharacters(in: .whitespaces)
guard !trimmed.isEmpty else { return connections }
return connections.filter { connection in
connection.displayName.localizedCaseInsensitiveContains(trimmed) ||
connection.host.localizedCaseInsensitiveContains(trimmed)
}
}
private func serviceLabel(_ raw: String) -> String {
switch raw {
case "_rfb._tcp": return "VNC (Screen Sharing)"
case "_workstation._tcp": return "Apple workstation"
default: return raw
}
}
private func prepareDiscoveredHost(_ host: DiscoveredHost) async {
resolvingHostID = host.id
defer { resolvingHostID = nil }
do {
let resolved = try await discovery.resolve(host)
addPrefill = AddConnectionPrefill(
displayName: host.displayName,
host: resolved.host,
port: resolved.port
)
} catch {
addPrefill = AddConnectionPrefill(
displayName: host.displayName,
host: "",
port: 5900
)
}
showingAdd = true
}
private func delete(_ connection: SavedConnection) {
try? KeychainService().deletePassword(account: connection.keychainTag)
modelContext.delete(connection)
try? modelContext.save()
}
}
struct SessionRoute: Hashable {
let connectionID: UUID
}

View File

@@ -2,36 +2,624 @@
import UIKit
import VNCCore
final class FramebufferUIView: UIView {
weak var coordinator: FramebufferView.Coordinator?
private let contentLayer = CALayer()
@MainActor
final class FramebufferUIView: UIView,
UIGestureRecognizerDelegate,
UIPointerInteractionDelegate,
UIKeyInput {
// MARK: - UITextInputTraits disable every kind of autocomplete, so
// each keystroke produces exactly one insertText() call and nothing
// is swallowed by the suggestion/predictive engine.
@objc var autocorrectionType: UITextAutocorrectionType = .no
@objc var autocapitalizationType: UITextAutocapitalizationType = .none
@objc var spellCheckingType: UITextSpellCheckingType = .no
@objc var smartQuotesType: UITextSmartQuotesType = .no
@objc var smartDashesType: UITextSmartDashesType = .no
@objc var smartInsertDeleteType: UITextSmartInsertDeleteType = .no
@objc var keyboardType: UIKeyboardType = .asciiCapable
@objc var keyboardAppearance: UIKeyboardAppearance = .dark
@objc var returnKeyType: UIReturnKeyType = .default
@objc var enablesReturnKeyAutomatically: Bool = false
@objc var isSecureTextEntry: Bool = false
@objc var passwordRules: UITextInputPasswordRules?
@objc var textContentType: UITextContentType? = UITextContentType(rawValue: "")
weak var controller: SessionController?
var inputMode: InputMode = .touch
var selectedScreen: RemoteScreen? {
didSet { setNeedsLayout() }
}
var zoomScale: CGFloat = 1.0 {
didSet { setNeedsLayout() }
}
var contentTranslation: CGPoint = .zero {
didSet { setNeedsLayout() }
}
var trackpadCursorNormalized: CGPoint = CGPoint(x: 0.5, y: 0.5)
var onTrackpadCursorChanged: ((CGPoint) -> Void)?
private let imageLayer = CALayer()
private let inputMapper = InputMapper()
// Touch-mode pan with implicit left-button drag
private var touchPanActiveButtonDown = false
private var touchPanLastNormalized: CGPoint?
// Trackpad-mode pan moves the soft cursor
private var trackpadPanStartCursor: CGPoint?
private var trackpadPanStartPoint: CGPoint?
// Two-finger scroll accumulator
private var scrollAccumulator: CGPoint = .zero
private static let scrollStepPoints: CGFloat = 28
// Pinch zoom anchor
private var pinchStartScale: CGFloat = 1.0
private var pinchStartTranslation: CGPoint = .zero
// Indirect pointer (trackpad/mouse via UIPointerInteraction)
private var indirectPointerNormalized: CGPoint?
// On-screen keyboard handling direct UIKeyInput conformance.
var onKeyboardDismissed: (() -> Void)?
private lazy var functionAccessoryView: UIView = makeFunctionAccessoryView()
private var keyboardWanted = false
// A separate accessibility element for test diagnostics, because iOS
// reserves `accessibilityValue` on UIKeyInput-classed views for text.
private let diagnosticProbe: UIView = {
let v = UIView(frame: .zero)
v.alpha = 0
v.isAccessibilityElement = true
v.accessibilityIdentifier = "fb-diag"
v.accessibilityLabel = ""
return v
}()
override init(frame: CGRect) {
super.init(frame: frame)
isOpaque = true
backgroundColor = .black
contentLayer.magnificationFilter = .nearest
contentLayer.minificationFilter = .linear
layer.addSublayer(contentLayer)
isMultipleTouchEnabled = true
clipsToBounds = true
imageLayer.magnificationFilter = .nearest
imageLayer.minificationFilter = .linear
imageLayer.contentsGravity = .resize
// Kill CALayer's implicit animations we want every framebuffer
// update to be an instantaneous blit, not a 0.25s crossfade.
imageLayer.actions = [
"contents": NSNull(),
"contentsRect": NSNull(),
"position": NSNull(),
"bounds": NSNull(),
"frame": NSNull(),
"transform": NSNull(),
"backgroundColor": NSNull()
]
layer.addSublayer(imageLayer)
configureGestureRecognizers()
configurePointerInteraction()
isAccessibilityElement = true
accessibilityLabel = "Remote framebuffer"
accessibilityIdentifier = "framebuffer"
addSubview(diagnosticProbe)
}
required init?(coder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
override func layoutSubviews() {
super.layoutSubviews()
contentLayer.frame = bounds
override var canBecomeFirstResponder: Bool { true }
override func didMoveToWindow() {
super.didMoveToWindow()
if window != nil { _ = becomeFirstResponder() }
}
func apply(state: SessionState) {
switch state {
case .connected(let size):
contentLayer.backgroundColor = UIColor.darkGray.cgColor
_ = size
default:
contentLayer.backgroundColor = UIColor.black.cgColor
override var inputAccessoryView: UIView? {
keyboardWanted ? functionAccessoryView : nil
}
/// Show or hide the iOS on-screen keyboard.
func setSoftwareKeyboardVisible(_ visible: Bool) {
appendDiagnostic("set:\(visible)")
if visible {
keyboardWanted = true
if !isFirstResponder {
let became = becomeFirstResponder()
appendDiagnostic("became:\(became)")
} else {
reloadInputViews()
appendDiagnostic("reload")
}
} else {
keyboardWanted = false
if isFirstResponder {
_ = resignFirstResponder()
appendDiagnostic("resigned")
}
}
}
private func appendDiagnostic(_ tag: String) {
let prior = diagnosticProbe.accessibilityLabel ?? ""
diagnosticProbe.accessibilityLabel = prior + "[\(tag)]"
}
// MARK: - UIKeyInput
var hasText: Bool { true }
func insertText(_ text: String) {
appendDiagnostic("ins:\(text)")
controller?.type(text)
}
func deleteBackward() {
appendDiagnostic("del")
controller?.sendBackspace()
}
// MARK: Layout / image
override func layoutSubviews() {
super.layoutSubviews()
applyLayerFrame()
}
func apply(image: CGImage?, framebufferSize: CGSize) {
CATransaction.begin()
CATransaction.setDisableActions(true)
imageLayer.contents = image
applyLayerFrame()
CATransaction.commit()
}
private func applyLayerFrame() {
let bounds = self.bounds
let fbSize = framebufferContentSize()
guard fbSize.width > 0, fbSize.height > 0,
bounds.width > 0, bounds.height > 0,
let displayed = inputMapper.displayedRect(for: fbSize, in: bounds.size) else {
imageLayer.frame = bounds
imageLayer.contentsRect = CGRect(x: 0, y: 0, width: 1, height: 1)
return
}
let scaled = displayed.applying(
CGAffineTransform(translationX: -bounds.midX, y: -bounds.midY)
.concatenating(CGAffineTransform(scaleX: zoomScale, y: zoomScale))
.concatenating(CGAffineTransform(translationX: bounds.midX + contentTranslation.x,
y: bounds.midY + contentTranslation.y))
)
imageLayer.frame = scaled
if let screen = selectedScreen {
let totalWidth = framebufferAbsoluteSize().width
let totalHeight = framebufferAbsoluteSize().height
if totalWidth > 0, totalHeight > 0 {
imageLayer.contentsRect = CGRect(
x: screen.frame.origin.x / totalWidth,
y: screen.frame.origin.y / totalHeight,
width: screen.frame.width / totalWidth,
height: screen.frame.height / totalHeight
)
} else {
imageLayer.contentsRect = CGRect(x: 0, y: 0, width: 1, height: 1)
}
} else {
imageLayer.contentsRect = CGRect(x: 0, y: 0, width: 1, height: 1)
}
}
private func framebufferAbsoluteSize() -> CGSize {
guard let size = controller?.framebufferSize else { return .zero }
return CGSize(width: CGFloat(size.width), height: CGFloat(size.height))
}
private func framebufferContentSize() -> CGSize {
if let screen = selectedScreen {
return CGSize(width: screen.frame.width, height: screen.frame.height)
}
return framebufferAbsoluteSize()
}
// MARK: Gesture recognition
private func configureGestureRecognizers() {
let singleTap = UITapGestureRecognizer(target: self, action: #selector(handleSingleTap(_:)))
singleTap.numberOfTapsRequired = 1
singleTap.numberOfTouchesRequired = 1
addGestureRecognizer(singleTap)
let twoFingerTap = UITapGestureRecognizer(target: self, action: #selector(handleTwoFingerTap(_:)))
twoFingerTap.numberOfTapsRequired = 1
twoFingerTap.numberOfTouchesRequired = 2
addGestureRecognizer(twoFingerTap)
let longPress = UILongPressGestureRecognizer(target: self, action: #selector(handleLongPress(_:)))
longPress.minimumPressDuration = 0.55
longPress.delegate = self
addGestureRecognizer(longPress)
let pan = UIPanGestureRecognizer(target: self, action: #selector(handlePan(_:)))
pan.minimumNumberOfTouches = 1
pan.maximumNumberOfTouches = 1
pan.delegate = self
pan.allowedTouchTypes = [
NSNumber(value: UITouch.TouchType.direct.rawValue),
NSNumber(value: UITouch.TouchType.pencil.rawValue)
]
addGestureRecognizer(pan)
let twoFingerPan = UIPanGestureRecognizer(target: self, action: #selector(handleTwoFingerPan(_:)))
twoFingerPan.minimumNumberOfTouches = 2
twoFingerPan.maximumNumberOfTouches = 2
twoFingerPan.delegate = self
addGestureRecognizer(twoFingerPan)
let pinch = UIPinchGestureRecognizer(target: self, action: #selector(handlePinch(_:)))
pinch.delegate = self
addGestureRecognizer(pinch)
let indirectScroll = UIPanGestureRecognizer(target: self, action: #selector(handleIndirectScroll(_:)))
indirectScroll.allowedScrollTypesMask = .all
indirectScroll.allowedTouchTypes = [NSNumber(value: UITouch.TouchType.indirectPointer.rawValue)]
indirectScroll.delegate = self
addGestureRecognizer(indirectScroll)
let hover = UIHoverGestureRecognizer(target: self, action: #selector(handleHover(_:)))
addGestureRecognizer(hover)
}
// MARK: UIGestureRecognizerDelegate
func gestureRecognizer(_ gestureRecognizer: UIGestureRecognizer,
shouldRecognizeSimultaneouslyWith other: UIGestureRecognizer) -> Bool {
if gestureRecognizer is UIPinchGestureRecognizer || other is UIPinchGestureRecognizer {
return true
}
return false
}
func gestureRecognizer(_ gestureRecognizer: UIGestureRecognizer,
shouldRequireFailureOf other: UIGestureRecognizer) -> Bool {
if let tap = gestureRecognizer as? UITapGestureRecognizer,
tap.numberOfTouchesRequired == 1,
let otherTap = other as? UITapGestureRecognizer,
otherTap.numberOfTouchesRequired == 2 {
return true
}
return false
}
// MARK: Pointer Interaction
private func configurePointerInteraction() {
let interaction = UIPointerInteraction(delegate: self)
addInteraction(interaction)
}
func pointerInteraction(_ interaction: UIPointerInteraction,
styleFor region: UIPointerRegion) -> UIPointerStyle? {
UIPointerStyle.hidden()
}
func pointerInteraction(_ interaction: UIPointerInteraction,
regionFor request: UIPointerRegionRequest,
defaultRegion: UIPointerRegion) -> UIPointerRegion? {
let viewPoint = request.location
if let normalized = inputMapper.normalize(viewPoint: viewPoint,
in: bounds.size,
framebufferSize: framebufferContentSize()) {
indirectPointerNormalized = normalized
controller?.pointerMove(toNormalized: mapToFullFramebuffer(normalized: normalized))
}
return defaultRegion
}
// MARK: Gesture Handlers
@objc private func handleSingleTap(_ recognizer: UITapGestureRecognizer) {
let location = recognizer.location(in: self)
switch inputMode {
case .touch:
if let normalized = normalizedFor(location) {
controller?.pointerClick(.left,
atNormalized: mapToFullFramebuffer(normalized: normalized))
}
case .trackpad:
controller?.pointerClick(.left,
atNormalized: mapToFullFramebuffer(normalized: trackpadCursorNormalized))
}
if !isFirstResponder { _ = becomeFirstResponder() }
}
@objc private func handleTwoFingerTap(_ recognizer: UITapGestureRecognizer) {
let location = recognizer.location(in: self)
switch inputMode {
case .touch:
if let normalized = normalizedFor(location) {
controller?.pointerClick(.right,
atNormalized: mapToFullFramebuffer(normalized: normalized))
}
case .trackpad:
controller?.pointerClick(.right,
atNormalized: mapToFullFramebuffer(normalized: trackpadCursorNormalized))
}
}
@objc private func handleLongPress(_ recognizer: UILongPressGestureRecognizer) {
guard recognizer.state == .began else { return }
let location = recognizer.location(in: self)
switch inputMode {
case .touch:
if let normalized = normalizedFor(location) {
controller?.pointerClick(.right,
atNormalized: mapToFullFramebuffer(normalized: normalized))
}
case .trackpad:
controller?.pointerClick(.right,
atNormalized: mapToFullFramebuffer(normalized: trackpadCursorNormalized))
}
}
@objc private func handlePan(_ recognizer: UIPanGestureRecognizer) {
switch inputMode {
case .touch:
handleTouchPan(recognizer)
case .trackpad:
handleTrackpadPan(recognizer)
}
}
private func handleTouchPan(_ recognizer: UIPanGestureRecognizer) {
let location = recognizer.location(in: self)
guard let normalized = normalizedFor(location) else { return }
let mapped = mapToFullFramebuffer(normalized: normalized)
switch recognizer.state {
case .began:
touchPanLastNormalized = mapped
touchPanActiveButtonDown = true
controller?.pointerDown(.left, atNormalized: mapped)
case .changed:
touchPanLastNormalized = mapped
controller?.pointerMove(toNormalized: mapped)
case .ended, .cancelled, .failed:
if touchPanActiveButtonDown {
touchPanActiveButtonDown = false
let endNormalized = touchPanLastNormalized ?? mapped
controller?.pointerUp(.left, atNormalized: endNormalized)
}
touchPanLastNormalized = nil
default:
break
}
}
private func handleTrackpadPan(_ recognizer: UIPanGestureRecognizer) {
let translation = recognizer.translation(in: self)
switch recognizer.state {
case .began:
trackpadPanStartCursor = trackpadCursorNormalized
trackpadPanStartPoint = .zero
case .changed:
guard let start = trackpadPanStartCursor else { return }
let fbSize = framebufferContentSize()
guard let displayed = inputMapper.displayedRect(for: fbSize, in: bounds.size),
displayed.width > 0, displayed.height > 0 else { return }
let dx = translation.x / displayed.width
let dy = translation.y / displayed.height
let newCursor = CGPoint(
x: max(0, min(1, start.x + dx)),
y: max(0, min(1, start.y + dy))
)
trackpadCursorNormalized = newCursor
onTrackpadCursorChanged?(newCursor)
controller?.pointerMove(toNormalized: mapToFullFramebuffer(normalized: newCursor))
case .ended, .cancelled, .failed:
trackpadPanStartCursor = nil
trackpadPanStartPoint = nil
default: break
}
}
@objc private func handleTwoFingerPan(_ recognizer: UIPanGestureRecognizer) {
let translation = recognizer.translation(in: self)
recognizer.setTranslation(.zero, in: self)
scrollAccumulator.x += translation.x
scrollAccumulator.y += translation.y
emitScrollEvents()
if recognizer.state == .ended || recognizer.state == .cancelled {
scrollAccumulator = .zero
}
}
@objc private func handleIndirectScroll(_ recognizer: UIPanGestureRecognizer) {
let translation = recognizer.translation(in: self)
recognizer.setTranslation(.zero, in: self)
scrollAccumulator.x += translation.x
scrollAccumulator.y += translation.y
emitScrollEvents()
if recognizer.state == .ended || recognizer.state == .cancelled {
scrollAccumulator = .zero
}
}
private func emitScrollEvents() {
guard let controller else { return }
let cursor: CGPoint = inputMode == .trackpad
? trackpadCursorNormalized
: (indirectPointerNormalized ?? CGPoint(x: 0.5, y: 0.5))
let mapped = mapToFullFramebuffer(normalized: cursor)
let stepsY = Int((scrollAccumulator.y / Self.scrollStepPoints).rounded(.towardZero))
if stepsY != 0 {
let dir: ScrollDirection = stepsY > 0 ? .down : .up
controller.pointerScroll(dir, steps: UInt32(abs(stepsY)), atNormalized: mapped)
scrollAccumulator.y -= CGFloat(stepsY) * Self.scrollStepPoints
}
let stepsX = Int((scrollAccumulator.x / Self.scrollStepPoints).rounded(.towardZero))
if stepsX != 0 {
let dir: ScrollDirection = stepsX > 0 ? .right : .left
controller.pointerScroll(dir, steps: UInt32(abs(stepsX)), atNormalized: mapped)
scrollAccumulator.x -= CGFloat(stepsX) * Self.scrollStepPoints
}
}
@objc private func handlePinch(_ recognizer: UIPinchGestureRecognizer) {
switch recognizer.state {
case .began:
pinchStartScale = zoomScale
pinchStartTranslation = contentTranslation
case .changed:
let newScale = max(0.5, min(8.0, pinchStartScale * recognizer.scale))
zoomScale = newScale
case .ended, .cancelled, .failed:
// snap back if zoomed too much
if zoomScale < 1.01 {
zoomScale = 1.0
contentTranslation = .zero
}
default: break
}
}
@objc private func handleHover(_ recognizer: UIHoverGestureRecognizer) {
let location = recognizer.location(in: self)
guard let normalized = inputMapper.normalize(viewPoint: location,
in: bounds.size,
framebufferSize: framebufferContentSize()) else { return }
indirectPointerNormalized = normalized
if recognizer.state == .changed || recognizer.state == .began {
controller?.pointerMove(toNormalized: mapToFullFramebuffer(normalized: normalized))
}
}
private func normalizedFor(_ point: CGPoint) -> CGPoint? {
inputMapper.normalize(viewPoint: point,
in: bounds.size,
framebufferSize: framebufferContentSize())
}
private func mapToFullFramebuffer(normalized: CGPoint) -> CGPoint {
guard let screen = selectedScreen,
let totalSize = controller?.framebufferSize,
totalSize.width > 0, totalSize.height > 0 else {
return normalized
}
let totalWidth = CGFloat(totalSize.width)
let totalHeight = CGFloat(totalSize.height)
let x = (screen.frame.origin.x + normalized.x * screen.frame.width) / totalWidth
let y = (screen.frame.origin.y + normalized.y * screen.frame.height) / totalHeight
return CGPoint(x: max(0, min(1, x)), y: max(0, min(1, y)))
}
// MARK: Hardware keyboard
override func pressesBegan(_ presses: Set<UIPress>, with event: UIPressesEvent?) {
var handled = false
for press in presses {
guard let key = press.key else { continue }
handleKey(key, isDown: true)
handled = true
}
if !handled { super.pressesBegan(presses, with: event) }
}
override func pressesEnded(_ presses: Set<UIPress>, with event: UIPressesEvent?) {
var handled = false
for press in presses {
guard let key = press.key else { continue }
handleKey(key, isDown: false)
handled = true
}
if !handled { super.pressesEnded(presses, with: event) }
}
override func pressesCancelled(_ presses: Set<UIPress>, with event: UIPressesEvent?) {
for press in presses {
guard let key = press.key else { continue }
handleKey(key, isDown: false)
}
super.pressesCancelled(presses, with: event)
}
private func handleKey(_ key: UIKey, isDown: Bool) {
guard let controller else { return }
let modifiers = KeyMapping.modifierKeysyms(for: key.modifierFlags)
if isDown {
for sym in modifiers { controller.keyDown(keysym: sym) }
}
if let sym = KeyMapping.keysym(for: key) {
if isDown {
controller.keyDown(keysym: sym)
} else {
controller.keyUp(keysym: sym)
}
}
if !isDown {
for sym in modifiers.reversed() { controller.keyUp(keysym: sym) }
}
}
// MARK: Software keyboard accessory
private func makeFunctionAccessoryView() -> UIView {
let bar = UIToolbar(frame: CGRect(x: 0, y: 0, width: UIScreen.main.bounds.width, height: 44))
bar.barStyle = .black
bar.tintColor = .white
bar.isTranslucent = true
func barButton(title: String?, image: String?, action: Selector) -> UIBarButtonItem {
if let title {
return UIBarButtonItem(title: title, style: .plain, target: self, action: action)
}
return UIBarButtonItem(image: UIImage(systemName: image ?? ""),
style: .plain,
target: self,
action: action)
}
let esc = barButton(title: "esc", image: nil, action: #selector(accessoryEsc))
let tab = barButton(title: "tab", image: nil, action: #selector(accessoryTab))
let ctrl = barButton(title: "ctrl", image: nil, action: #selector(accessoryControl))
let cmd = barButton(title: "", image: nil, action: #selector(accessoryCommand))
let opt = barButton(title: "", image: nil, action: #selector(accessoryOption))
let left = barButton(title: nil, image: "arrow.left", action: #selector(accessoryLeft))
let down = barButton(title: nil, image: "arrow.down", action: #selector(accessoryDown))
let up = barButton(title: nil, image: "arrow.up", action: #selector(accessoryUp))
let right = barButton(title: nil, image: "arrow.right", action: #selector(accessoryRight))
let spacer = UIBarButtonItem.flexibleSpace()
let dismiss = barButton(title: nil,
image: "keyboard.chevron.compact.down",
action: #selector(accessoryDismiss))
bar.items = [esc, tab, ctrl, cmd, opt, spacer, left, down, up, right, spacer, dismiss]
return bar
}
@objc private func accessoryEsc() { controller?.sendEscape() }
@objc private func accessoryTab() { controller?.sendTab() }
@objc private func accessoryControl() {
controller?.pressKeyCombo([.control])
}
@objc private func accessoryCommand() {
controller?.pressKeyCombo([.command])
}
@objc private func accessoryOption() {
controller?.pressKeyCombo([.option])
}
@objc private func accessoryLeft() { controller?.sendArrow(.left) }
@objc private func accessoryDown() { controller?.sendArrow(.down) }
@objc private func accessoryUp() { controller?.sendArrow(.up) }
@objc private func accessoryRight() { controller?.sendArrow(.right) }
@objc private func accessoryDismiss() {
setSoftwareKeyboardVisible(false)
}
}
#endif

View File

@@ -1,36 +1,63 @@
#if canImport(UIKit)
import SwiftUI
import VNCCore
import CoreGraphics
struct FramebufferView: UIViewRepresentable {
let controller: SessionController
let inputMode: InputMode
let selectedScreen: RemoteScreen?
@Binding var trackpadCursor: CGPoint
@Binding var showSoftwareKeyboard: Bool
func makeUIView(context: Context) -> FramebufferUIView {
let view = FramebufferUIView()
view.coordinator = context.coordinator
view.controller = controller
view.inputMode = inputMode
view.selectedScreen = selectedScreen
view.trackpadCursorNormalized = trackpadCursor
view.onTrackpadCursorChanged = { [binding = $trackpadCursor] new in
binding.wrappedValue = new
}
view.onKeyboardDismissed = { [binding = $showSoftwareKeyboard] in
if binding.wrappedValue { binding.wrappedValue = false }
}
return view
}
func updateUIView(_ uiView: FramebufferUIView, context: Context) {
uiView.apply(state: controller.state)
uiView.controller = controller
uiView.inputMode = inputMode
uiView.selectedScreen = selectedScreen
if uiView.trackpadCursorNormalized != trackpadCursor {
uiView.trackpadCursorNormalized = trackpadCursor
}
uiView.apply(image: controller.currentImage,
framebufferSize: framebufferSize)
uiView.setSoftwareKeyboardVisible(showSoftwareKeyboard)
// Touch the revision so SwiftUI re-runs us when frames arrive
_ = controller.imageRevision
}
func makeCoordinator() -> Coordinator {
Coordinator(inputMapper: InputMapper())
}
@MainActor
final class Coordinator {
let inputMapper: InputMapper
init(inputMapper: InputMapper) { self.inputMapper = inputMapper }
private var framebufferSize: CGSize {
if let size = controller.framebufferSize {
return CGSize(width: CGFloat(size.width), height: CGFloat(size.height))
}
return .zero
}
}
#else
import SwiftUI
import VNCCore
import CoreGraphics
struct FramebufferView: View {
let controller: SessionController
let inputMode: InputMode
let selectedScreen: RemoteScreen?
@Binding var trackpadCursor: CGPoint
@Binding var showSoftwareKeyboard: Bool
var body: some View { Color.black }
}
#endif

View File

@@ -1,30 +1,73 @@
import CoreGraphics
import Foundation
public enum InputMode: Sendable {
public enum InputMode: String, Sendable, CaseIterable, Hashable {
case touch
case trackpad
}
public struct PointerEvent: Sendable, Equatable {
public let location: CGPoint
public let buttonMask: UInt8
}
public struct DisplayedRect: Equatable, Sendable {
public let rect: CGRect
public struct KeyEvent: Sendable, Equatable {
public let keysym: UInt32
public let down: Bool
}
@MainActor
public final class InputMapper {
public var mode: InputMode = .touch
public init() {}
public func pointerFromTap(at point: CGPoint, in framebuffer: CGSize, viewBounds: CGSize) -> PointerEvent {
let x = point.x / viewBounds.width * framebuffer.width
let y = point.y / viewBounds.height * framebuffer.height
return PointerEvent(location: CGPoint(x: x, y: y), buttonMask: 0b1)
public init(rect: CGRect) {
self.rect = rect
}
}
public struct InputMapper: Sendable {
public init() {}
/// Return the rect where a framebuffer of `framebufferSize` is drawn inside
/// `viewSize` using aspect-fit (`.resizeAspect`) gravity.
public func displayedRect(for framebufferSize: CGSize,
in viewSize: CGSize) -> CGRect? {
guard framebufferSize.width > 0, framebufferSize.height > 0,
viewSize.width > 0, viewSize.height > 0 else {
return nil
}
let viewAspect = viewSize.width / viewSize.height
let fbAspect = framebufferSize.width / framebufferSize.height
if viewAspect > fbAspect {
let displayedWidth = viewSize.height * fbAspect
let xOffset = (viewSize.width - displayedWidth) / 2
return CGRect(x: xOffset, y: 0,
width: displayedWidth,
height: viewSize.height)
} else {
let displayedHeight = viewSize.width / fbAspect
let yOffset = (viewSize.height - displayedHeight) / 2
return CGRect(x: 0, y: yOffset,
width: viewSize.width,
height: displayedHeight)
}
}
/// Convert a point in view coordinates to normalized framebuffer coordinates [0,1].
/// Returns nil if either size is empty.
public func normalize(viewPoint: CGPoint,
in viewSize: CGSize,
framebufferSize: CGSize) -> CGPoint? {
guard let displayed = displayedRect(for: framebufferSize, in: viewSize) else {
return nil
}
let nx = (viewPoint.x - displayed.origin.x) / displayed.width
let ny = (viewPoint.y - displayed.origin.y) / displayed.height
return CGPoint(x: clamp(nx), y: clamp(ny))
}
/// Convert a normalized framebuffer point [0,1] to view coordinates.
public func viewPoint(forNormalized normalized: CGPoint,
in viewSize: CGSize,
framebufferSize: CGSize) -> CGPoint? {
guard let displayed = displayedRect(for: framebufferSize, in: viewSize) else {
return nil
}
let x = displayed.origin.x + normalized.x * displayed.width
let y = displayed.origin.y + normalized.y * displayed.height
return CGPoint(x: x, y: y)
}
private func clamp(_ value: CGFloat) -> CGFloat {
max(0, min(1, value))
}
}

View File

@@ -0,0 +1,114 @@
#if canImport(UIKit)
import UIKit
enum KeyMapping {
/// X11 keysym values for special keys.
enum X11 {
static let backspace: UInt32 = 0xFF08
static let tab: UInt32 = 0xFF09
static let `return`: UInt32 = 0xFF0D
static let escape: UInt32 = 0xFF1B
static let delete: UInt32 = 0xFFFF
static let home: UInt32 = 0xFF50
static let leftArrow: UInt32 = 0xFF51
static let upArrow: UInt32 = 0xFF52
static let rightArrow: UInt32 = 0xFF53
static let downArrow: UInt32 = 0xFF54
static let pageUp: UInt32 = 0xFF55
static let pageDown: UInt32 = 0xFF56
static let end: UInt32 = 0xFF57
static let insert: UInt32 = 0xFF63
static let f1: UInt32 = 0xFFBE
static let f2: UInt32 = 0xFFBF
static let f3: UInt32 = 0xFFC0
static let f4: UInt32 = 0xFFC1
static let f5: UInt32 = 0xFFC2
static let f6: UInt32 = 0xFFC3
static let f7: UInt32 = 0xFFC4
static let f8: UInt32 = 0xFFC5
static let f9: UInt32 = 0xFFC6
static let f10: UInt32 = 0xFFC7
static let f11: UInt32 = 0xFFC8
static let f12: UInt32 = 0xFFC9
static let leftShift: UInt32 = 0xFFE1
static let rightShift: UInt32 = 0xFFE2
static let leftControl: UInt32 = 0xFFE3
static let rightControl: UInt32 = 0xFFE4
static let capsLock: UInt32 = 0xFFE5
static let leftAlt: UInt32 = 0xFFE9
static let rightAlt: UInt32 = 0xFFEA
static let leftMeta: UInt32 = 0xFFE7
static let rightMeta: UInt32 = 0xFFE8
static let leftCommand: UInt32 = 0xFFEB
static let rightCommand: UInt32 = 0xFFEC
static let space: UInt32 = 0x0020
}
/// Convert a UIKey into one or more VNC keysyms (UInt32 X11 values).
/// Returns nil if no mapping is found.
static func keysym(for key: UIKey) -> UInt32? {
let usage = key.keyCode
if let mapped = mapHIDUsage(usage) {
return mapped
}
let chars = key.charactersIgnoringModifiers
guard let scalar = chars.unicodeScalars.first else { return nil }
return scalar.value
}
static func modifierKeysyms(for flags: UIKeyModifierFlags) -> [UInt32] {
var out: [UInt32] = []
if flags.contains(.shift) { out.append(X11.leftShift) }
if flags.contains(.control) { out.append(X11.leftControl) }
if flags.contains(.alternate) { out.append(X11.leftAlt) }
if flags.contains(.command) { out.append(X11.leftCommand) }
return out
}
private static func mapHIDUsage(_ usage: UIKeyboardHIDUsage) -> UInt32? {
switch usage {
case .keyboardReturnOrEnter, .keypadEnter: return X11.return
case .keyboardEscape: return X11.escape
case .keyboardDeleteOrBackspace: return X11.backspace
case .keyboardTab: return X11.tab
case .keyboardSpacebar: return X11.space
case .keyboardLeftArrow: return X11.leftArrow
case .keyboardRightArrow: return X11.rightArrow
case .keyboardUpArrow: return X11.upArrow
case .keyboardDownArrow: return X11.downArrow
case .keyboardHome: return X11.home
case .keyboardEnd: return X11.end
case .keyboardPageUp: return X11.pageUp
case .keyboardPageDown: return X11.pageDown
case .keyboardInsert: return X11.insert
case .keyboardDeleteForward: return X11.delete
case .keyboardCapsLock: return X11.capsLock
case .keyboardLeftShift: return X11.leftShift
case .keyboardRightShift: return X11.rightShift
case .keyboardLeftControl: return X11.leftControl
case .keyboardRightControl: return X11.rightControl
case .keyboardLeftAlt: return X11.leftAlt
case .keyboardRightAlt: return X11.rightAlt
case .keyboardLeftGUI: return X11.leftCommand
case .keyboardRightGUI: return X11.rightCommand
case .keyboardF1: return X11.f1
case .keyboardF2: return X11.f2
case .keyboardF3: return X11.f3
case .keyboardF4: return X11.f4
case .keyboardF5: return X11.f5
case .keyboardF6: return X11.f6
case .keyboardF7: return X11.f7
case .keyboardF8: return X11.f8
case .keyboardF9: return X11.f9
case .keyboardF10: return X11.f10
case .keyboardF11: return X11.f11
case .keyboardF12: return X11.f12
default: return nil
}
}
}
#endif

View File

@@ -0,0 +1,105 @@
import SwiftUI
import VNCCore
struct SessionToolbar: View {
let controller: SessionController
@Binding var inputMode: InputMode
@Binding var showKeyboardBar: Bool
@Binding var selectedScreenID: UInt32?
var onScreenshot: () -> Void
var onDisconnect: () -> Void
var body: some View {
HStack(spacing: 10) {
modePicker
.frame(maxWidth: 180)
iconButton(systemName: "keyboard",
label: "Toggle keyboard bar",
isOn: showKeyboardBar) {
showKeyboardBar.toggle()
}
if controller.screens.count > 1 {
screenMenu
}
iconButton(systemName: "camera",
label: "Take screenshot") {
onScreenshot()
}
Spacer(minLength: 0)
iconButton(systemName: "xmark",
label: "Disconnect",
tint: .red) {
onDisconnect()
}
}
.padding(.horizontal, 14)
.padding(.vertical, 8)
.glassSurface(in: Capsule())
.padding(.horizontal, 12)
}
private var modePicker: some View {
Picker("Input mode", selection: $inputMode) {
Image(systemName: "hand.tap.fill")
.accessibilityLabel("Touch")
.tag(InputMode.touch)
Image(systemName: "rectangle.and.hand.point.up.left.filled")
.accessibilityLabel("Trackpad")
.tag(InputMode.trackpad)
}
.pickerStyle(.segmented)
}
private var screenMenu: some View {
Menu {
Button {
selectedScreenID = nil
} label: {
Label("All screens",
systemImage: selectedScreenID == nil ? "checkmark" : "rectangle.on.rectangle")
}
ForEach(controller.screens) { screen in
Button {
selectedScreenID = screen.id
} label: {
Label(
"Screen \(String(screen.id)) \(Int(screen.frame.width))×\(Int(screen.frame.height))",
systemImage: selectedScreenID == screen.id ? "checkmark" : "display"
)
}
}
} label: {
iconBadge(systemName: "rectangle.on.rectangle", tint: .primary)
}
.accessibilityLabel("Choose monitor")
}
private func iconButton(systemName: String,
label: String,
isOn: Bool = false,
tint: Color = .primary,
action: @escaping () -> Void) -> some View {
Button(action: action) {
iconBadge(systemName: systemName, tint: tint, isOn: isOn)
}
.accessibilityLabel(label)
.accessibilityIdentifier(label)
.buttonStyle(.plain)
}
private func iconBadge(systemName: String, tint: Color = .primary, isOn: Bool = false) -> some View {
Image(systemName: systemName)
.font(.callout.weight(.semibold))
.foregroundStyle(tint == .red ? Color.red : tint)
.frame(width: 34, height: 34)
.background(
Circle().fill(isOn ? tint.opacity(0.20) : Color.clear)
)
.contentShape(Rectangle())
}
}

View File

@@ -1,9 +1,26 @@
import SwiftUI
import SwiftData
import VNCCore
import CoreGraphics
#if canImport(UIKit)
import UIKit
#endif
public struct SessionView: View {
let connection: SavedConnection
@Environment(\.modelContext) private var modelContext
@Environment(\.dismiss) private var dismiss
@Environment(\.scenePhase) private var scenePhase
@AppStorage("defaultInputMode") private var defaultInputModeRaw = "touch"
@State private var controller: SessionController?
@State private var inputMode: InputMode = .touch
@State private var showSoftwareKeyboard = false
@State private var trackpadCursor: CGPoint = CGPoint(x: 0.5, y: 0.5)
@State private var selectedScreenID: UInt32?
@State private var screenshotItem: ScreenshotShareItem?
@State private var chromeVisible = true
public init(connection: SavedConnection) {
self.connection = connection
@@ -12,57 +29,330 @@ public struct SessionView: View {
public var body: some View {
ZStack {
Color.black.ignoresSafeArea()
// Hidden accessibility probe for UI tests; reflects SwiftUI
// state so we can verify binding propagation.
Color.clear
.frame(width: 1, height: 1)
.accessibilityIdentifier("sessionview-state")
.accessibilityLabel("kb=\(showSoftwareKeyboard)")
if let controller {
FramebufferView(controller: controller)
statusOverlay(for: controller.state)
FramebufferView(
controller: controller,
inputMode: inputMode,
selectedScreen: selectedScreen(for: controller),
trackpadCursor: $trackpadCursor,
showSoftwareKeyboard: $showSoftwareKeyboard
)
.ignoresSafeArea()
.onTapGesture(count: 3) {
withAnimation(.snappy(duration: 0.22)) { chromeVisible.toggle() }
}
if inputMode == .trackpad,
let size = controller.framebufferSize {
TrackpadCursorOverlay(
normalizedPosition: trackpadCursor,
framebufferSize: CGSize(width: CGFloat(size.width),
height: CGFloat(size.height)),
isVisible: true
)
}
statusOverlay(for: controller.state, controller: controller)
if chromeVisible {
overlayChrome(controller: controller)
.transition(.opacity.combined(with: .move(edge: .top)))
}
// Persistent pull-tab so the chrome is always reachable
// (three-finger tap is the power-user shortcut).
VStack {
chromeToggleHandle
Spacer()
}
} else {
ProgressView("Preparing session…")
.tint(.white)
.foregroundStyle(.white)
}
}
.navigationTitle(connection.displayName)
.navigationTitle(controller?.desktopName ?? connection.displayName)
#if os(iOS)
.navigationBarTitleDisplayMode(.inline)
.toolbar(.hidden, for: .navigationBar)
.toolbar(.hidden, for: .tabBar)
#endif
.task(id: connection.id) {
await startSession()
}
.onChange(of: scenePhase) { _, phase in
if phase == .background {
controller?.stop()
}
}
.onDisappear {
controller?.stop()
persistLastConnected()
}
.sheet(item: $screenshotItem) { item in
ShareSheet(items: [item.image])
}
}
private var chromeToggleHandle: some View {
Button {
withAnimation(.snappy(duration: 0.22)) {
chromeVisible.toggle()
}
} label: {
Image(systemName: chromeVisible ? "chevron.up" : "chevron.down")
.font(.caption.weight(.bold))
.foregroundStyle(.white)
.frame(width: 44, height: 20)
.contentShape(Rectangle())
}
.buttonStyle(.plain)
.glassSurface(in: Capsule())
.accessibilityLabel(chromeVisible ? "Hide toolbar" : "Show toolbar")
.padding(.top, chromeVisible ? 4 : 2)
}
@ViewBuilder
private func statusOverlay(for state: SessionState) -> some View {
private func overlayChrome(controller: SessionController) -> some View {
VStack(spacing: 8) {
HStack(spacing: 10) {
Button {
stopAndDismiss()
} label: {
Image(systemName: "chevron.left")
.font(.callout.weight(.semibold))
.frame(width: 34, height: 34)
}
.buttonStyle(.plain)
.glassSurface(in: Circle())
.accessibilityLabel("Back")
connectionLabel(controller: controller)
if controller.viewOnly {
HStack(spacing: 4) {
Image(systemName: "eye.fill")
.font(.caption2)
Text("View only")
.font(.caption.weight(.semibold))
}
.foregroundStyle(.yellow)
.padding(.horizontal, 10)
.padding(.vertical, 5)
.glassSurface(in: Capsule())
}
Spacer()
}
.padding(.horizontal, 12)
.padding(.top, 4)
SessionToolbar(
controller: controller,
inputMode: $inputMode,
showKeyboardBar: $showSoftwareKeyboard,
selectedScreenID: $selectedScreenID,
onScreenshot: { takeScreenshot(controller: controller) },
onDisconnect: { stopAndDismiss() }
)
Spacer()
}
}
@ViewBuilder
private func connectionLabel(controller: SessionController) -> some View {
VStack(alignment: .leading, spacing: 0) {
Text(controller.desktopName ?? connection.displayName)
.font(.headline)
.foregroundStyle(.white)
.lineLimit(1)
if let size = controller.framebufferSize {
Text("\(size.width)×\(size.height)")
.font(.caption2.monospacedDigit())
.foregroundStyle(.white.opacity(0.6))
}
}
.padding(.horizontal, 14)
.padding(.vertical, 6)
.glassSurface(in: Capsule())
}
@ViewBuilder
private func statusOverlay(for state: SessionState,
controller: SessionController) -> some View {
switch state {
case .connecting:
VStack {
messageOverlay {
ProgressView("Connecting…").tint(.white).foregroundStyle(.white)
}
case .authenticating:
VStack {
messageOverlay {
ProgressView("Authenticating…").tint(.white).foregroundStyle(.white)
}
case .disconnected(let reason):
VStack(spacing: 8) {
Image(systemName: "exclamationmark.triangle.fill")
.font(.largeTitle)
Text("Disconnected")
.font(.headline)
Text(String(describing: reason))
.font(.caption)
.foregroundStyle(.secondary)
}
.foregroundStyle(.white)
default:
disconnectedOverlay(reason: reason, controller: controller)
case .idle, .connected:
EmptyView()
}
}
@MainActor
@ViewBuilder
private func disconnectedOverlay(reason: DisconnectReason,
controller: SessionController) -> some View {
VStack(spacing: 14) {
Image(systemName: glyph(for: reason))
.font(.system(size: 36, weight: .semibold))
.foregroundStyle(tint(for: reason))
Text(headline(for: reason))
.font(.headline)
.foregroundStyle(.white)
Text(humanReadable(reason: reason, lastError: controller.lastErrorMessage))
.font(.callout)
.foregroundStyle(.white.opacity(0.75))
.multilineTextAlignment(.center)
.padding(.horizontal, 24)
HStack(spacing: 12) {
Button {
controller.reconnectNow()
} label: {
Label("Reconnect", systemImage: "arrow.clockwise")
}
.glassButton(prominent: true)
Button("Close") {
stopAndDismiss()
}
.glassButton()
}
if controller.isReconnecting {
HStack(spacing: 8) {
ProgressView().controlSize(.small)
Text("Retrying… attempt \(controller.reconnectAttempt)")
.font(.caption)
.foregroundStyle(.white.opacity(0.7))
}
}
}
.padding(28)
.frame(maxWidth: 360)
.glassSurface(in: RoundedRectangle(cornerRadius: 28, style: .continuous))
}
@ViewBuilder
private func messageOverlay<V: View>(@ViewBuilder _ content: () -> V) -> some View {
content()
.padding(22)
.glassSurface(in: RoundedRectangle(cornerRadius: 22, style: .continuous))
}
private func selectedScreen(for controller: SessionController) -> RemoteScreen? {
guard let id = selectedScreenID else { return nil }
return controller.screens.first { $0.id == id }
}
private func startSession() async {
let endpoint = TransportEndpoint(host: connection.host, port: connection.port)
let transport = DirectTransport(endpoint: endpoint)
let controller = SessionController(transport: transport)
self.controller = controller
controller.start()
if controller == nil {
inputMode = preferredInputMode()
let provider = DefaultPasswordProvider()
let controller = SessionController(connection: connection,
passwordProvider: provider)
self.controller = controller
controller.start()
}
}
private func preferredInputMode() -> InputMode {
switch connection.inputMode {
case .trackpad: return .trackpad
case .touch: return InputMode(rawValue: defaultInputModeRaw) ?? .touch
}
}
private func stopAndDismiss() {
controller?.stop()
persistLastConnected()
dismiss()
}
private func persistLastConnected() {
connection.lastConnectedAt = .now
try? modelContext.save()
}
private func humanReadable(reason: DisconnectReason, lastError: String?) -> String {
switch reason {
case .userRequested: return "You ended this session."
case .authenticationFailed: return "Authentication failed. Check the password and try again."
case .networkError(let detail): return lastError ?? detail
case .protocolError(let detail): return lastError ?? detail
case .remoteClosed: return "The remote computer closed the session."
}
}
private func glyph(for reason: DisconnectReason) -> String {
switch reason {
case .userRequested: "checkmark.circle"
case .authenticationFailed: "lock.trianglebadge.exclamationmark"
case .networkError, .remoteClosed: "wifi.exclamationmark"
case .protocolError: "exclamationmark.triangle"
}
}
private func tint(for reason: DisconnectReason) -> Color {
switch reason {
case .userRequested: .green
case .authenticationFailed: .yellow
default: .orange
}
}
private func headline(for reason: DisconnectReason) -> String {
switch reason {
case .userRequested: "Session ended"
case .authenticationFailed: "Authentication failed"
default: "Disconnected"
}
}
private func takeScreenshot(controller: SessionController) {
#if canImport(UIKit)
guard let cgImage = controller.currentImage else { return }
let image = UIImage(cgImage: cgImage)
screenshotItem = ScreenshotShareItem(image: image)
#endif
}
}
private struct ScreenshotShareItem: Identifiable {
let id = UUID()
#if canImport(UIKit)
let image: UIImage
#else
let image: Any
#endif
}
#if canImport(UIKit)
private struct ShareSheet: UIViewControllerRepresentable {
let items: [Any]
func makeUIViewController(context: Context) -> UIActivityViewController {
UIActivityViewController(activityItems: items, applicationActivities: nil)
}
func updateUIViewController(_ uiViewController: UIActivityViewController, context: Context) {}
}
#else
private struct ShareSheet: View {
let items: [Any]
var body: some View { EmptyView() }
}
#endif

View File

@@ -0,0 +1,85 @@
import SwiftUI
import VNCCore
struct SoftKeyboardBar: View {
let controller: SessionController
@Binding var isExpanded: Bool
var body: some View {
VStack(spacing: 8) {
HStack(spacing: 8) {
key("esc", wide: false) { controller.sendEscape() }
key("tab", wide: false) { controller.sendTab() }
key("", wide: false) { controller.sendReturn() }
iconKey("delete.left") { controller.sendBackspace() }
Spacer(minLength: 6)
arrowCluster
Button {
withAnimation(.snappy(duration: 0.22)) { isExpanded.toggle() }
} label: {
Image(systemName: isExpanded ? "chevron.down.circle.fill" : "fn")
.font(.callout.weight(.semibold))
}
.buttonStyle(.plain)
.frame(width: 36, height: 32)
.accessibilityLabel(isExpanded ? "Collapse function keys" : "Expand function keys")
}
if isExpanded {
HStack(spacing: 6) {
ForEach(1...12, id: \.self) { idx in
Button("F\(idx)") {
controller.sendFunctionKey(idx)
}
.buttonStyle(.plain)
.font(.caption.weight(.medium))
.frame(maxWidth: .infinity)
.frame(height: 28)
.background(
RoundedRectangle(cornerRadius: 8, style: .continuous)
.fill(Color.primary.opacity(0.08))
)
}
}
}
}
.padding(.horizontal, 12)
.padding(.vertical, 10)
.glassSurface(in: RoundedRectangle(cornerRadius: 22, style: .continuous))
.padding(.horizontal, 12)
}
private func key(_ label: String, wide: Bool, action: @escaping () -> Void) -> some View {
Button(action: action) {
Text(label)
.font(.callout.weight(.medium))
.frame(width: wide ? 64 : 44, height: 32)
.background(
RoundedRectangle(cornerRadius: 10, style: .continuous)
.fill(Color.primary.opacity(0.10))
)
}
.buttonStyle(.plain)
}
private func iconKey(_ systemName: String, action: @escaping () -> Void) -> some View {
Button(action: action) {
Image(systemName: systemName)
.font(.callout.weight(.medium))
.frame(width: 44, height: 32)
.background(
RoundedRectangle(cornerRadius: 10, style: .continuous)
.fill(Color.primary.opacity(0.10))
)
}
.buttonStyle(.plain)
}
private var arrowCluster: some View {
HStack(spacing: 4) {
iconKey("arrow.left") { controller.sendArrow(.left) }
iconKey("arrow.up") { controller.sendArrow(.up) }
iconKey("arrow.down") { controller.sendArrow(.down) }
iconKey("arrow.right") { controller.sendArrow(.right) }
}
}
}

View File

@@ -0,0 +1,29 @@
import SwiftUI
import VNCCore
struct TrackpadCursorOverlay: View {
let normalizedPosition: CGPoint
let framebufferSize: CGSize
let isVisible: Bool
var body: some View {
GeometryReader { proxy in
let mapper = InputMapper()
if isVisible,
let displayed = mapper.displayedRect(for: framebufferSize, in: proxy.size),
displayed.width > 0 && displayed.height > 0 {
let x = displayed.origin.x + normalizedPosition.x * displayed.width
let y = displayed.origin.y + normalizedPosition.y * displayed.height
Image(systemName: "cursorarrow")
.font(.system(size: 22, weight: .bold))
.foregroundStyle(.white, .black)
.shadow(radius: 1)
.position(x: x, y: y)
.allowsHitTesting(false)
.accessibilityHidden(true)
.transition(.opacity)
}
}
.ignoresSafeArea()
}
}

View File

@@ -2,28 +2,120 @@ import SwiftUI
import VNCCore
public struct SettingsView: View {
@AppStorage("clipboardSyncEnabled") private var clipboardSync = true
@Environment(\.dismiss) private var dismiss
@AppStorage("clipboardSyncEnabled") private var clipboardSyncDefault = true
@AppStorage("defaultInputMode") private var defaultInputModeRaw = "touch"
@AppStorage("autoReconnectEnabled") private var autoReconnect = true
@AppStorage("reduceMotionInSession") private var reduceMotion = false
public init() {}
public var body: some View {
NavigationStack {
Form {
Section("Input") {
Section {
HStack(spacing: 14) {
ZStack {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.fill(LinearGradient(
colors: [.blue, .purple],
startPoint: .topLeading,
endPoint: .bottomTrailing
))
.frame(width: 56, height: 56)
Image(systemName: "display")
.font(.title.weight(.semibold))
.foregroundStyle(.white)
}
VStack(alignment: .leading, spacing: 2) {
Text("Screens")
.font(.headline)
Text("Version \(Self.shortVersion) (\(Self.buildNumber))")
.font(.caption)
.foregroundStyle(.secondary)
}
Spacer()
}
.padding(.vertical, 4)
}
Section {
Picker("Default input mode", selection: $defaultInputModeRaw) {
Text("Touch").tag("touch")
Text("Trackpad").tag("trackpad")
}
} header: {
Text("Input")
} footer: {
Text("Each saved connection can override this.")
}
Section("Privacy") {
Toggle("Sync clipboard with remote", isOn: $clipboardSync)
Section {
Toggle("Auto-reconnect on drop", isOn: $autoReconnect)
} header: {
Text("Connection")
} footer: {
Text("Retries with jittered exponential backoff up to ~30 seconds.")
}
Section("About") {
LabeledContent("Version", value: "0.1 (Phase 0)")
Section {
Toggle("Sync clipboard with remote", isOn: $clipboardSyncDefault)
} header: {
Text("Privacy")
} footer: {
Text("Each connection can override. Passwords are stored in iOS Keychain on this device only — they never sync to iCloud.")
}
Section {
Toggle("Reduce motion in session", isOn: $reduceMotion)
} header: {
Text("Display")
}
Section {
Link(destination: URL(string: "https://example.com/privacy")!) {
Label("Privacy policy", systemImage: "hand.raised")
}
Link(destination: URL(string: "https://github.com/royalapplications/royalvnc")!) {
Label("RoyalVNCKit on GitHub", systemImage: "link")
}
} header: {
Text("About")
}
}
#if os(iOS)
.scrollContentBackground(.hidden)
#endif
.background(formBackground.ignoresSafeArea())
.navigationTitle("Settings")
#if os(iOS)
.navigationBarTitleDisplayMode(.inline)
#endif
.toolbar {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
}
}
private var formBackground: some View {
LinearGradient(
colors: [
Color(red: 0.04, green: 0.05, blue: 0.10),
Color(red: 0.02, green: 0.02, blue: 0.05)
],
startPoint: .top,
endPoint: .bottom
)
}
private static var shortVersion: String {
Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "0.0"
}
private static var buildNumber: String {
Bundle.main.infoDictionary?["CFBundleVersion"] as? String ?? "0"
}
}

View File

@@ -0,0 +1,64 @@
import SwiftUI
extension View {
/// Applies the iOS 26 Liquid Glass effect when available, falling back to
/// `ultraThinMaterial` on earlier OSes. Pass a clipping shape so the glass
/// has the right silhouette.
@ViewBuilder
func glassSurface<S: Shape>(in shape: S = RoundedRectangle(cornerRadius: 22, style: .continuous)) -> some View {
if #available(iOS 26.0, macOS 26.0, *) {
self.glassEffect(.regular, in: shape)
} else {
self
.background(.ultraThinMaterial, in: shape)
.overlay(shape.stroke(Color.white.opacity(0.08), lineWidth: 1))
}
}
/// Interactive version of `glassSurface` pressed states animate.
@ViewBuilder
func interactiveGlassSurface<S: Shape>(in shape: S = Capsule()) -> some View {
if #available(iOS 26.0, macOS 26.0, *) {
self.glassEffect(.regular.interactive(), in: shape)
} else {
self
.background(.thinMaterial, in: shape)
.overlay(shape.stroke(Color.white.opacity(0.10), lineWidth: 1))
}
}
/// Replaces a button's chrome with a Liquid Glass capsule when available.
@ViewBuilder
func glassButton(prominent: Bool = false) -> some View {
if #available(iOS 26.0, macOS 26.0, *) {
if prominent {
self.buttonStyle(.glassProminent)
} else {
self.buttonStyle(.glass)
}
} else {
if prominent {
self.buttonStyle(.borderedProminent)
} else {
self.buttonStyle(.bordered)
}
}
}
/// Soft top-edge fade so content slips under the floating toolbar without
/// a hard cutoff.
@ViewBuilder
func softScrollEdge() -> some View {
if #available(iOS 26.0, macOS 26.0, *) {
self.scrollEdgeEffectStyle(.soft, for: .top)
} else {
self
}
}
}
/// Pretty-prints a port number without locale grouping ("5900", not "5,900").
@inlinable
func portLabel(_ port: Int) -> String {
String(port)
}

View File

@@ -3,13 +3,53 @@ import Testing
import CoreGraphics
@Suite struct InputMapperTests {
@Test @MainActor func tapInMiddleMapsToFramebufferCenter() {
@Test func centerOfViewMapsToCenterOfFramebuffer() {
let mapper = InputMapper()
let fb = CGSize(width: 1920, height: 1080)
let view = CGSize(width: 192, height: 108)
let event = mapper.pointerFromTap(at: CGPoint(x: 96, y: 54), in: fb, viewBounds: view)
#expect(event.location.x == 960)
#expect(event.location.y == 540)
#expect(event.buttonMask == 0b1)
let normalized = mapper.normalize(viewPoint: CGPoint(x: 96, y: 54),
in: view,
framebufferSize: fb)
#expect(normalized != nil)
#expect(abs((normalized?.x ?? 0) - 0.5) < 0.001)
#expect(abs((normalized?.y ?? 0) - 0.5) < 0.001)
}
@Test func aspectFitLetterboxesTallerView() {
let mapper = InputMapper()
let fb = CGSize(width: 1600, height: 900) // 16:9
let view = CGSize(width: 800, height: 800) // 1:1
let displayed = mapper.displayedRect(for: fb, in: view)
#expect(displayed != nil)
#expect(abs((displayed?.width ?? 0) - 800) < 0.001)
#expect(abs((displayed?.height ?? 0) - 450) < 0.001)
#expect(abs((displayed?.origin.y ?? 0) - 175) < 0.001)
}
@Test func aspectFitPillarboxesWiderView() {
let mapper = InputMapper()
let fb = CGSize(width: 800, height: 800)
let view = CGSize(width: 1600, height: 900)
let displayed = mapper.displayedRect(for: fb, in: view)
#expect(displayed != nil)
#expect(abs((displayed?.height ?? 0) - 900) < 0.001)
#expect(abs((displayed?.width ?? 0) - 900) < 0.001)
#expect(abs((displayed?.origin.x ?? 0) - 350) < 0.001)
}
@Test func roundTripNormalizationIsStable() {
let mapper = InputMapper()
let fb = CGSize(width: 1920, height: 1080)
let view = CGSize(width: 800, height: 600)
let target = CGPoint(x: 0.25, y: 0.75)
let viewPoint = mapper.viewPoint(forNormalized: target,
in: view,
framebufferSize: fb)
let normalized = mapper.normalize(viewPoint: viewPoint ?? .zero,
in: view,
framebufferSize: fb)
#expect(normalized != nil)
#expect(abs((normalized?.x ?? 0) - target.x) < 0.001)
#expect(abs((normalized?.y ?? 0) - target.y) < 0.001)
}
}

View File

@@ -1,6 +1,6 @@
name: Screens
options:
bundleIdPrefix: com.example.screens
bundleIdPrefix: com.tt.screens
deploymentTarget:
iOS: "18.0"
developmentLanguage: en
@@ -11,6 +11,7 @@ settings:
SWIFT_STRICT_CONCURRENCY: complete
ENABLE_USER_SCRIPT_SANDBOXING: YES
CODE_SIGN_STYLE: Automatic
DEVELOPMENT_TEAM: V3PF3M6B6U
packages:
VNCCore:
@@ -25,15 +26,35 @@ targets:
deploymentTarget: "18.0"
sources:
- path: Screens
excludes:
- Resources/Info.plist
- path: Screens/Resources/PrivacyInfo.xcprivacy
type: file
buildPhase: resources
settings:
base:
PRODUCT_BUNDLE_IDENTIFIER: com.example.screens
PRODUCT_BUNDLE_IDENTIFIER: com.tt.screens
PRODUCT_NAME: Screens
TARGETED_DEVICE_FAMILY: "1,2"
GENERATE_INFOPLIST_FILE: NO
INFOPLIST_FILE: Screens/Resources/Info.plist
SWIFT_EMIT_LOC_STRINGS: YES
dependencies:
- package: VNCCore
product: VNCCore
- package: VNCUI
product: VNCUI
ScreensUITests:
type: bundle.ui-testing
platform: iOS
deploymentTarget: "18.0"
sources:
- path: ScreensUITests
settings:
base:
PRODUCT_BUNDLE_IDENTIFIER: com.tt.screens.uitests
TEST_TARGET_NAME: Screens
GENERATE_INFOPLIST_FILE: YES
dependencies:
- target: Screens

View File

@@ -1,17 +1,64 @@
import SwiftUI
import SwiftData
import VNCCore
import VNCUI
@main
struct VNCApp: App {
@State private var appState = AppStateController()
private let sharedContainer: ModelContainer = {
let schema = Schema([SavedConnection.self])
let cloudKitConfiguration = ModelConfiguration(
"CloudConnections",
schema: schema,
cloudKitDatabase: .automatic
)
if let container = try? ModelContainer(for: schema,
configurations: [cloudKitConfiguration]) {
return container
}
let local = ModelConfiguration(
"LocalConnections",
schema: schema,
cloudKitDatabase: .none
)
return (try? ModelContainer(for: schema, configurations: [local]))
?? (try! ModelContainer(for: SavedConnection.self))
}()
var body: some Scene {
WindowGroup {
RootView()
.environment(appState)
.task { await appState.initialize() }
}
.modelContainer(for: SavedConnection.self)
.modelContainer(sharedContainer)
WindowGroup("Session", for: UUID.self) { $connectionID in
DetachedSessionWindow(connectionID: connectionID)
}
.modelContainer(sharedContainer)
}
}
private struct DetachedSessionWindow: View {
let connectionID: UUID?
@Environment(\.modelContext) private var modelContext
@Query private var connections: [SavedConnection]
var body: some View {
if let id = connectionID,
let match = connections.first(where: { $0.id == id }) {
NavigationStack {
SessionView(connection: match)
}
} else {
ContentUnavailableView(
"Connection unavailable",
systemImage: "questionmark.app",
description: Text("This window's connection is no longer available.")
)
}
}
}

View File

@@ -0,0 +1,14 @@
{
"images" : [
{
"filename" : "icon-1024.png",
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 200 KiB

View File

@@ -0,0 +1,6 @@
{
"info" : {
"author" : "xcode",
"version" : 1
}
}

View File

@@ -14,16 +14,22 @@
<string>6.0</string>
<key>CFBundleName</key>
<string>$(PRODUCT_NAME)</string>
<key>CFBundleIconName</key>
<string>AppIcon</string>
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>0.1</string>
<string>0.4</string>
<key>CFBundleVersion</key>
<string>1</string>
<string>4</string>
<key>LSRequiresIPhoneOS</key>
<true/>
<key>NSLocalNetworkUsageDescription</key>
<string>Discover computers on your network that you can control remotely.</string>
<key>NSPasteboardUsageDescription</key>
<string>Sync the clipboard between this device and the remote computer when you opt in.</string>
<key>NSCameraUsageDescription</key>
<string>Optionally capture a frame of the remote screen.</string>
<key>NSBonjourServices</key>
<array>
<string>_rfb._tcp</string>
@@ -34,6 +40,13 @@
<key>UIApplicationSupportsMultipleScenes</key>
<true/>
</dict>
<key>UILaunchScreen</key>
<dict>
<key>UIColorName</key>
<string></string>
</dict>
<key>UISupportsDocumentBrowser</key>
<false/>
<key>UIRequiredDeviceCapabilities</key>
<array>
<string>arm64</string>
@@ -51,5 +64,7 @@
<string>UIInterfaceOrientationLandscapeLeft</string>
<string>UIInterfaceOrientationLandscapeRight</string>
</array>
<key>UIBackgroundModes</key>
<array/>
</dict>
</plist>

View File

@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSPrivacyTracking</key>
<false/>
<key>NSPrivacyTrackingDomains</key>
<array/>
<key>NSPrivacyCollectedDataTypes</key>
<array/>
<key>NSPrivacyAccessedAPITypes</key>
<array>
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategoryUserDefaults</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<array>
<string>CA92.1</string>
</array>
</dict>
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategoryFileTimestamp</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<array>
<string>C617.1</string>
</array>
</dict>
<dict>
<key>NSPrivacyAccessedAPIType</key>
<string>NSPrivacyAccessedAPICategorySystemBootTime</string>
<key>NSPrivacyAccessedAPITypeReasons</key>
<array>
<string>35F9.1</string>
</array>
</dict>
</array>
</dict>
</plist>

View File

@@ -0,0 +1,141 @@
import XCTest
final class ScreensUITests: XCTestCase {
override func setUp() {
continueAfterFailure = false
}
@MainActor
func testLayoutFillsScreenAndCoreFlows() {
let app = XCUIApplication()
app.launch()
// ---- Top chrome is present and flush with the safe area top.
let title = app.staticTexts["Screens"]
XCTAssertTrue(title.waitForExistence(timeout: 5), "Title should appear")
let screenFrame = app.windows.firstMatch.frame
let titleY = title.frame.midY
XCTAssertLessThan(titleY, screenFrame.height * 0.25,
"Title should sit in the top 25% of the screen; actual Y \(titleY) in screen \(screenFrame.height)")
// ---- Tap + to open Add Connection sheet.
let addButton = app.buttons["Add connection"]
XCTAssertTrue(addButton.exists, "Add button should exist")
addButton.tap()
let displayNameField = app.textFields["Display name"]
XCTAssertTrue(displayNameField.waitForExistence(timeout: 2),
"Add Connection sheet should present and show Display name field")
app.buttons["Cancel"].tap()
XCTAssertFalse(displayNameField.waitForExistence(timeout: 1),
"Add sheet should dismiss on Cancel")
// ---- Settings gear opens Settings sheet.
app.buttons["Settings"].tap()
let settingsTitle = app.navigationBars.staticTexts["Settings"]
XCTAssertTrue(settingsTitle.waitForExistence(timeout: 2),
"Settings sheet should present")
app.buttons["Done"].tap()
// ---- Search field accepts input and clears.
let search = app.textFields.matching(NSPredicate(format: "placeholderValue == %@", "Search connections")).firstMatch
XCTAssertTrue(search.waitForExistence(timeout: 2), "Search field should exist")
search.tap()
search.typeText("mini")
XCTAssertEqual(search.value as? String, "mini",
"Search text should round-trip")
XCTAssertTrue(title.isHittable, "Title should remain on screen during search")
// ---- Empty-state CTA routes to Add Connection.
let emptyCTA = app.buttons["Add a computer"]
if emptyCTA.waitForExistence(timeout: 1) {
app.buttons["Clear search"].tap()
emptyCTA.tap()
XCTAssertTrue(displayNameField.waitForExistence(timeout: 2),
"Empty-state CTA should present Add Connection sheet")
app.buttons["Cancel"].tap()
}
}
/// Adds a connection with an unreachable host so the SessionView opens
/// (controller exists, no real network needed) and verifies that tapping
/// the keyboard icon presents the iOS keyboard and that pressing keys
/// flows through the framebuffer view's input handling.
@MainActor
func testSoftwareKeyboardSendsCharactersToFramebuffer() {
let app = XCUIApplication()
app.launch()
// Add a bogus connection
app.buttons["Add connection"].tap()
let nameField = app.textFields["Display name"]
XCTAssertTrue(nameField.waitForExistence(timeout: 2))
nameField.tap()
nameField.typeText("KeyboardTest")
let hostField = app.textFields["Host or IP"]
hostField.tap()
hostField.typeText("127.0.0.1")
app.buttons["Add"].tap()
// Open the connection
app.buttons.matching(NSPredicate(format: "label CONTAINS[c] %@", "KeyboardTest")).firstMatch.tap()
// Framebuffer exists (as a TextView once UIKeyInput is adopted)
let framebuffer = app.descendants(matching: .any)
.matching(identifier: "framebuffer").firstMatch
XCTAssertTrue(framebuffer.waitForExistence(timeout: 5),
"Framebuffer view should exist in session")
// The diagnostic probe is a hidden sibling element that records
// keyboard plumbing events via its accessibilityLabel.
let diag = app.descendants(matching: .any)
.matching(identifier: "fb-diag").firstMatch
XCTAssertTrue(diag.waitForExistence(timeout: 3),
"Diagnostic probe should exist")
// State probe
let state = app.descendants(matching: .any)
.matching(identifier: "sessionview-state").firstMatch
XCTAssertTrue(state.waitForExistence(timeout: 3))
XCTAssertEqual(state.label, "kb=false", "Initial binding should be false")
// Tap the keyboard toggle in the toolbar
let kbToggle = app.buttons["Toggle keyboard bar"]
XCTAssertTrue(kbToggle.waitForExistence(timeout: 2))
kbToggle.tap()
// Verify the binding flipped
let flippedTrue = NSPredicate(format: "label == %@", "kb=true")
wait(for: [expectation(for: flippedTrue, evaluatedWith: state)], timeout: 3)
// Wait for the diagnostic to record that we asked for the keyboard.
// The framebuffer was already first responder (for hardware keys), so
// we hit the reload path rather than `became:true`, which is fine.
let askedForKeyboard = NSPredicate(format: "label CONTAINS %@", "[set:true]")
wait(for: [expectation(for: askedForKeyboard, evaluatedWith: diag)],
timeout: 3)
// Force the on-screen keyboard to appear (simulator suppresses it
// when Connect Hardware Keyboard is on).
// The simulator can be told to disconnect the host keyboard via the
// hardware menu; we instead just wait briefly and then attempt taps.
Thread.sleep(forTimeInterval: 0.5)
// If the system keyboard is visible, drive it like a user. Otherwise
// fall back to typeText (which bypasses the UI keyboard).
let kb = app.keyboards.firstMatch
if kb.waitForExistence(timeout: 1) {
kb.keys["h"].tap()
kb.keys["i"].tap()
} else {
app.typeText("hi")
}
let typedH = NSPredicate(format: "label CONTAINS %@", "[ins:h]")
wait(for: [expectation(for: typedH, evaluatedWith: diag)], timeout: 3)
let typedI = NSPredicate(format: "label CONTAINS %@", "[ins:i]")
wait(for: [expectation(for: typedI, evaluatedWith: diag)], timeout: 3)
}
}

85
docs/HANDOFF.md Normal file
View File

@@ -0,0 +1,85 @@
# Handoff — Screens VNC app
A snapshot of project state so another machine (or another Claude Code session) can pick up without any out-of-band context.
## Resume on a new machine
```sh
# 1. Clone
git clone git@gitea.treytartt.com:admin/Screens.git
cd Screens
# 2. Prerequisites (macOS host)
brew install xcodegen # reproducible .xcodeproj generation
# Xcode 16+ required (Xcode 26 tested)
# 3. Generate the Xcode project (not committed)
xcodegen generate
# 4. Open in Xcode (or build from CLI)
open Screens.xcodeproj
# - or -
xcodebuild -scheme Screens -destination 'platform=iOS Simulator,name=iPhone 17' build
# 5. Fast unit tests (no simulator)
cd Packages/VNCCore && swift test # ~0.4s, 3 tests
cd ../VNCUI && swift test # ~3s, 1 test
```
## Git layout
- `main` — cumulative integrated history. Default branch.
- `phase-N-<topic>` — per-phase branches; open a PR into `main` and merge.
- Current branch in progress: **`phase-1-vnc-wiring`** (branched from `main` @ Phase 0 scaffold).
## Phase status
| Phase | Scope | Status |
|-------|-------|--------|
| 0 — Scaffold | Packages, app target, xcodegen, RoyalVNCKit dep, CI-compilable | ✅ merged to `main` (commit `2cff17f`) |
| 1 — MVP connect/view/tap | Wire `VNCConnectionDelegate`, render framebuffer, touch input, Keychain auth | 🔄 in progress on `phase-1-vnc-wiring` |
| 2 — Input parity | Trackpad mode, hardware keyboard, pointer, adaptive quality, reconnect | ⏳ not started |
| 3 — Productivity | Clipboard sync, multi-monitor, Apple Pencil, screenshots, view-only, curtain mode | ⏳ not started |
| 4 — Polish & ship | iPad multi-window, CloudKit sync, a11y, privacy manifest, TestFlight prep | ⏳ not started |
Full plan: [docs/PLAN.md](./PLAN.md).
## Critical architectural notes
1. **RoyalVNCKit owns its own networking.** It constructs its own `NWConnection` internally from `VNCConnection.Settings`. Our `Transport` protocol in `VNCCore` is *not* on the RFB path — it's kept as the extension point for future SSH tunneling. Don't try to pipe bytes from `DirectTransport` into `VNCConnection`.
2. **Delegate model.** `SessionController` should conform to `VNCConnectionDelegate` and bridge delegate callbacks (which arrive on internal RoyalVNCKit queues) to `@MainActor` state. Password supplied via `credentialFor:completion:` callback.
3. **iOS framebuffer rendering is on us.** RoyalVNCKit ships macOS-only `VNCFramebufferView` (NSView). We render by grabbing `framebuffer.cgImage` and setting `FramebufferUIView.contentLayer.contents` on each `didUpdateFramebuffer` callback. IOSurface-backed path available for zero-copy later.
4. **Swift 6 strict concurrency is on.** Cross-actor hops need care. Use `@MainActor` on everything UI-adjacent; mark Transport-style types as actors.
5. **Name "Screens" is owned by Edovia.** Pick a different App Store title before any public artifact.
## Dependencies
| Package | Purpose | Version | License |
|---------|---------|---------|---------|
| [royalvnc](https://github.com/royalapplications/royalvnc) | RFB protocol, encodings, auth | `branch: main` (tagged releases blocked by transitive CryptoSwift unstable-version constraint) | MIT |
| Apple first-party | Network, SwiftData, Security, UIKit, SwiftUI, Observation | iOS 18 SDK | — |
## Build artifacts
- `Screens.xcodeproj/`**git-ignored**. Regenerate with `xcodegen generate`.
- `Packages/*/.build/` — git-ignored. `swift build` or Xcode resolves.
- `Packages/*/Package.resolved` — git-ignored; change if you want reproducible dep versions across machines (recommended for an app target — consider flipping later).
## Files to look at first
- `docs/PLAN.md` — full plan
- `Project.yml` — xcodegen project definition
- `Screens/App/AppStateController.swift` — app-level state machine
- `Packages/VNCCore/Sources/VNCCore/Session/SessionController.swift` — the stub that Phase 1 will replace with a real `VNCConnectionDelegate` integration
- `Packages/VNCUI/Sources/VNCUI/Session/FramebufferUIView.swift` — where incoming framebuffer `CGImage`s land
## Open decisions
- **App Store name** — not "Screens" (trademarked).
- **Bundle identifier** — currently `com.example.screens` placeholder; set to your real prefix.
- **Team ID / signing** — currently `CODE_SIGN_STYLE: Automatic`; point at your team for device builds.
- **Privacy manifest** — scheduled for Phase 4; will enumerate `UIPasteboard`, `UserDefaults`, `NSPrivacyAccessedAPICategoryFileTimestamp` reasons.

232
docs/PLAN.md Normal file
View File

@@ -0,0 +1,232 @@
# Plan: VNC Client App (Screens-style) — iPhone + iPad
## Context
The user wants to build a VNC remote-desktop app for iPhone and iPad modeled on [Screens by Edovia](https://www.edovia.com/en/screens/), which is the market leader in the category. Screens exposes a premium, polished VNC experience: Bonjour discovery, saved connections, local + SSH tunneled transport, Touch Mode / Trackpad Mode input, clipboard sync, multi-monitor, hardware keyboard + Apple Pencil, and (in the full product) a relay service and file transfer.
Working directory `/Users/treyt/Desktop/code/Screens` is empty — this is a greenfield build. Scope agreed with the user:
- **Platforms**: iPhone + iPad (iOS/iPadOS). No Mac / visionOS in phase 1.
- **RFB engine**: [RoyalVNCKit](https://github.com/royalapplications/royalvnc) (MIT, Swift, supports Raw/CopyRect/RRE/CoRRE/Hextile/Zlib/ZRLE/Tight + VNC password + Apple Remote Desktop auth). iOS UIView rendering is flagged "work in progress" upstream, so we own the UIKit framebuffer view.
- **MVP ambition**: "Feature-parity push" — MVP + clipboard sync, multi-monitor, trackpad mode, Apple Pencil. ~2 months.
- **Remote access strategy**: Tailscale. A device on a tailnet is reached by its Tailscale IP or MagicDNS name exactly like a LAN host, and the transport is already WireGuard-encrypted end-to-end. The app does not need to bundle SSH tunneling or a relay — we just connect TCP and let Tailscale handle the network layer.
- **No account system / subscription** in phase 1. Saved connections sync via CloudKit (iCloud) if the user opts in.
Intended outcome: a shippable, App Store-quality VNC client that a Mac/Linux/RPi user can drive from an iPhone or iPad on their LAN or through SSH.
## Architecture
```
┌──────────────────────────────────────────────────────────────┐
│ App target (thin shell, ~10% of code) │
│ VNCApp.swift · RootView · AppStateController │
└──────────────────────┬───────────────────────────────────────┘
│ environment
┌──────────────────────▼───────────────────────────────────────┐
│ VNCUI (Swift package) │
│ ConnectionListView · AddConnectionView · SessionView │
│ FramebufferView (UIViewRepresentable → FramebufferUIView) │
│ InputMapper (Touch/Trackpad/HW keyboard/Pencil) │
└──────────────────────┬───────────────────────────────────────┘
┌──────────────────────▼───────────────────────────────────────┐
│ VNCCore (Swift package, @testable without simulator) │
│ SessionController ◀─────── Transport protocol │
│ │ └── DirectTransport (NW) │
│ ▼ │
│ RoyalVNCKit · DiscoveryService (NWBrowser _rfb._tcp) │
│ ConnectionStore (SwiftData + Keychain refs) │
│ KeychainService · ClipboardBridge │
└──────────────────────────────────────────────────────────────┘
```
`Transport` stays behind a protocol so a future relay or optional SSH-tunnel add-on can slot in without touching `SessionController`.
### Why this shape
- **Thin `@main`** and `AppStateController`: the app has distinct states (launching → list → connecting → connected → error). Modeling them as an enum avoids "boolean soup" and keeps logic testable. Pattern from the `axiom:axiom-app-composition` skill (Part 1).
- **VNCCore is a Swift Package** with no UI imports. This lets us run the protocol + transport tests with `swift test` (~sub-second) instead of `xcodebuild test`. 60× faster TDD loop.
- **Transport protocol** isolates the network layer from the session layer. Phase-1 has one implementation (`DirectTransport` over `NWConnection`), which already handles both LAN and Tailscale — a tailnet host resolves and routes like any other host from the app's perspective. If we ever add SSH tunneling or a rendezvous relay, they drop in as additional `Transport` conformers.
- **UIKit-backed framebuffer view** wrapped in `UIViewRepresentable`. SwiftUI's `Canvas` is too slow for 30+ fps blits at ~2M pixels. A `CALayer` with `contents = IOSurface`-backed `CGImage` (or a small Metal view if ZRLE/Tight decode runs on GPU later) gives us consistent 60 Hz.
## Key Dependencies
| Package | Purpose | License |
| --- | --- | --- |
| [royalvnc](https://github.com/royalapplications/royalvnc) | RFB protocol, encodings, auth | MIT |
| Apple first-party: `Network`, `SwiftData`, `CloudKit`, `Security` (Keychain), `UIKit`, `SwiftUI`, `Combine` | n/a | n/a |
Tailscale requires no SDK integration — it runs as a system-wide VPN/Network Extension installed separately by the user. Once active, remote hosts are reachable by Tailscale IP or MagicDNS name from our plain `NWConnection`.
Deployment target: **iOS 18.0**. Gets us `@Observable` macro, SwiftData stability fixes, and modern Network.framework APIs. Skip iOS 26-only features (Liquid Glass components) behind `if #available` gates so we don't block on Xcode 26 availability.
## Data Model
SwiftData models live in `VNCCore`:
```swift
@Model final class SavedConnection {
@Attribute(.unique) var id: UUID
var displayName: String
var host: String // hostname, IPv4, or IPv6
var port: Int // default 5900
var colorTag: ColorTag // UI identification
var lastConnectedAt: Date?
var preferredEncodings: [String] // ordered, persisted for reconnect
var keychainTag: String // opaque ref to VNC password entry
var quality: QualityPreset // .adaptive / .high / .low
var viewOnly: Bool
var curtainMode: Bool
}
```
VNC passwords go to Keychain (`kSecClassGenericPassword` with `kSecAttrAccessible = kSecAttrAccessibleWhenUnlockedThisDeviceOnly`). The `keychainTag` on the model is the account name — we never persist the secret in SwiftData. Optional CloudKit sync of `SavedConnection` (minus secrets) via SwiftData's CloudKit integration.
## Phased Delivery
### Phase 0 — Scaffold (~3 days)
- Create Xcode project (working name — App Store name TBD to avoid trademark conflict; brainstorm candidates before public launch).
- SPM packages `VNCCore` and `VNCUI` inside the repo.
- Add RoyalVNCKit as a dependency.
- Thin `@main`, `AppStateController` with `case launching / list / connecting(SavedConnection) / session(SessionController) / error(AppError)`, `RootView` switch.
- CI via `xcodebuild test` on `VNCUI` + `swift test` on `VNCCore`.
- App icon + launch screen placeholder.
### Phase 1 — MVP connect + view + tap (~2 weeks)
1. `DiscoveryService` using `NWBrowser` on `_rfb._tcp` and `_workstation._tcp` (macOS Screen Sharing advertises the latter).
2. `ConnectionStore` — SwiftData CRUD + Keychain round-trip.
3. `ConnectionListView`, `AddConnectionView`, basic Liquid Glass cards when available.
4. `DirectTransport` using `NWConnection` (TCP, TLS optional for VeNCrypt later).
5. `SessionController` wrapping `RoyalVNCKit.VNCConnection`: lifecycle, framebuffer updates, reconnection.
6. `FramebufferUIView` (UIKit): `CALayer` whose `contents` is a `CGImage` built from the RoyalVNC framebuffer bytes. Handle partial rect updates (dirty-rect blits, not full redraws).
7. Touch Mode input: tap → left click, two-finger tap → right click, pan → scroll. Nothing fancy yet.
8. VNC password auth + "save to Keychain" flow.
**Exit criterion**: from a cold launch I can discover my Mac via Bonjour, enter the Screen Sharing password, see the desktop, click a Finder icon, and type into Spotlight.
### Phase 2 — Input parity (~1.5 weeks)
- Trackpad Mode: a floating soft cursor SwiftUI overlay; pan moves the cursor, tap-to-click, pinch to zoom into the framebuffer. Momentum scroll.
- Hardware keyboard via `pressesBegan`/`pressesEnded` on the framebuffer view — full modifier + function key support (⌘, ⌥, ⌃, ⇧, F1F20, arrows).
- Magic Trackpad 2 / Magic Mouse: `UIPointerInteraction` + indirect pointer events → VNC pointer events.
- Adaptive quality: choose encoding (Tight vs ZRLE) and JPEG quality based on `NWPathMonitor` link type + measured RTT.
- Reconnect backoff with jitter.
### Phase 3 — Productivity features (~2 weeks)
- Clipboard sync: bidirectional `UIPasteboard` ↔ VNC `ClientCutText`/`ServerCutText`. Handle Unicode (ISO-8859-1 limitation of classic spec — prefer extended clipboard pseudo-encoding where the server advertises it). Opt-in toggle per connection and a global kill switch for privacy.
- Multi-monitor: parse `DesktopSize` / `ExtendedDesktopSize` pseudo-encodings; offer a monitor picker that sends `SetDesktopSize` or pans the viewport. Remember last selection per connection.
- Apple Pencil: treat as pointer with pressure — if the remote is a Mac running a drawing app, pass pressure via a custom extension if available, otherwise plain pointer. Hover support on M2 iPad Pro.
- Screenshot capture: grab current framebuffer bytes → `CGImage` → share sheet.
- "On disconnect" actions: prompt / reconnect / return to list.
- View Only and Curtain Mode toggles wired into RoyalVNC's session options.
### Phase 4 — Polish + ship (~1.5 weeks)
- iPad multi-window: `WindowGroup(for: SavedConnection.ID.self)` so each connection can open in its own window, each with its own `SessionController` instance.
- CloudKit sync of `SavedConnection` records via SwiftData's `ModelConfiguration(cloudKitDatabase: .private("iCloud..."))`. Secrets stay local-Keychain only.
- Accessibility pass (VoiceOver, Dynamic Type on non-framebuffer chrome, Reduce Motion).
- Empty states, error recovery, network-change handling (Wi-Fi → cellular with session pause/resume).
- Privacy manifest, App Store screenshots, TestFlight.
## Critical Files to Create
```
Screens.xcodeproj
Screens/
App/
VNCApp.swift # @main (thin)
RootView.swift
AppStateController.swift
Resources/
Assets.xcassets
Info.plist
Packages/VNCCore/
Package.swift
Sources/VNCCore/
Session/
SessionController.swift # wraps RoyalVNCKit.VNCConnection
SessionState.swift # enum of connection states
Transport/
Transport.swift # protocol
DirectTransport.swift # NWConnection
SSHTunnelTransport.swift # SwiftNIOSSH
Discovery/
DiscoveryService.swift # NWBrowser on _rfb._tcp + _workstation._tcp
Storage/
SavedConnection.swift # @Model
ConnectionStore.swift # actor-wrapped ModelContext helper
Security/
KeychainService.swift # SecItem wrapper
Clipboard/
ClipboardBridge.swift
Tests/VNCCoreTests/
TransportTests.swift
SessionControllerTests.swift # mock Transport
ConnectionStoreTests.swift
Packages/VNCUI/
Package.swift
Sources/VNCUI/
List/
ConnectionListView.swift
ConnectionCard.swift
Edit/
AddConnectionView.swift
Session/
SessionView.swift # hosts the framebuffer + toolbars
FramebufferView.swift # UIViewRepresentable
FramebufferUIView.swift # UIKit CALayer-backed renderer
InputMapper.swift # touch/trackpad/keyboard → RFB events
TrackpadCursorOverlay.swift
ToolbarView.swift
Settings/
SettingsView.swift
```
## Reusable Pieces (don't rebuild)
- **RoyalVNCKit** gives us the entire RFB state machine, encodings, auth. We *consume* it; we do not fork.
- **`NWBrowser` + `NWListener`** from Network.framework — no need for third-party Bonjour.
- **`AppStateController` pattern** from `axiom:axiom-app-composition` — copy the boilerplate for loading/list/session/error states.
- **`axiom:axiom-swiftui-nav`** for the iPad split-view connection-list-to-session navigation.
- **`axiom:axiom-uikit-bridging`** for `UIViewRepresentable` coordinator patterns on `FramebufferUIView`.
- **`axiom:axiom-keychain`** for the `SecItem` wrapper.
- **`axiom:axiom-networking`** for `NWConnection` TLS/retry patterns reused inside `DirectTransport`.
## Verification
Layered testing, unit-heavy because the simulator is slow.
1. **VNCCore unit tests (`swift test`, ~sub-second)**
- `TransportTests`: feed a synthetic RFB handshake byte stream through a `MockTransport`, assert `SessionController` transitions.
- `ConnectionStoreTests`: in-memory SwiftData container, verify CRUD + Keychain round-trip (Keychain stubbed via protocol in tests).
- `DiscoveryServiceTests`: inject a fake `NWBrowser` surface, verify published result list.
2. **VNCUI UI snapshot / interaction tests (`xcodebuild test`)**
- `InputMapperTests`: given a `CGPoint` in framebuffer coordinates and a gesture, assert the emitted RFB pointer/key events.
- Preview-driven manual checks for `ConnectionListView`, `AddConnectionView` on iPhone + iPad size classes.
3. **End-to-end manual matrix** (run before TestFlight each phase):
- Mac (macOS Screen Sharing, port 5900) — discovery, password, Touch Mode click through, clipboard.
- Linux (`tigervnc-server` in Docker) — ZRLE encoding, resolution change during session.
- Raspberry Pi (`realvnc-vnc-server`) — Raspberry Pi Connect auth variant, low-bandwidth adaptive quality.
- Tailscale remote case: the same three above with the iPhone on cellular reaching the host by its MagicDNS name.
- iPad multi-window: two simultaneous sessions to different hosts.
- Cellular → Wi-Fi handoff mid-session (airplane-mode toggle).
4. **Performance budget** — on iPhone 15, a 2560×1440 @ 30 fps ZRLE stream should stay under 25% CPU and 40% battery-per-hour. Profile with Instruments Time Profiler + Energy Log each phase. See `axiom:axiom-performance-profiling`.
5. **App Review preflight**: privacy manifest for `NSUserDefaults`, `UIPasteboard`, and network access; screenshots per device class; `axiom:axiom-app-store-submission` checklist before first TestFlight.
## Open Questions to Resolve Before Phase 0
- **App Store name**: "Screens" is owned by Edovia — we cannot ship under that name. Pick a working title (e.g. "Pikelet", "Portal VNC", "Farcast") before public artifacts.
- **Bundle identifier + team ID**: confirm Apple Developer account + agreed bundle prefix.
- **App icon + brand direction**: can be deferred to Phase 5 but worth sketching early.
## Sources
- [Screens by Edovia — marketing site](https://www.edovia.com/en/screens/)
- [RoyalVNCKit (royalapplications/royalvnc)](https://github.com/royalapplications/royalvnc)
- [RFB protocol — Wikipedia](https://en.wikipedia.org/wiki/RFB_protocol) · [RFC 6143 (RFB 3.8)](https://datatracker.ietf.org/doc/html/rfc6143)
- [Tailscale on iOS](https://tailscale.com/kb/1020/install-ios)
- Axiom skills consulted: `axiom-app-composition`, `axiom-uikit-bridging`, `axiom-networking`, `axiom-keychain`, `axiom-swiftui-nav`, `axiom-performance-profiling`, `axiom-app-store-submission`.

21
icons/PHILOSOPHY.md Normal file
View File

@@ -0,0 +1,21 @@
# Luminous Distance
A visual philosophy for objects that exist between presence and absence — interfaces that reach across space without disturbing the stillness at either end.
## Manifesto
Luminous Distance treats remote control not as mastery but as quiet correspondence. A signal leaves one room and arrives in another. Between those rooms is glass: cold, smooth, and aware of itself. The icon is a fragment of that glass — thick enough to hold a scene, thin enough to hint at what's behind it. Depth without decoration.
Color operates as atmosphere, not ornament. Palettes move in slow gradations — indigo deepening into ultramarine, cyan cooling into navy, warm magentas burning at the edges of cool blues — the way a display looks when you stand in a dark room and let your eyes adjust. Saturation is restrained where it matters and allowed to swell only at the focal point, producing a single core of light the eye can't escape. Nothing shouts. Everything glows.
Form is geometric but never brittle. Rounded rectangles hold the composition at every scale — a formal homage to the devices we touch all day. Within those frames, geometry is subdivided with the patience of draftsmanship: centimeter margins, optical centering, subtle tilts that suggest perspective without committing to 3D. Silhouettes are strong enough to read at a thumbnail's distance and rich enough to reward close inspection. Every curve is a deliberate choice, the product of painstaking correction and re-correction.
Material is drawn, not faked. Liquid Glass is not a filter; it is the literal subject — refraction, specular highlights, the faint chromatic bloom at the edge where a surface meets light. Where texture appears, it is the texture of something real: the phosphor glow of an old CRT, the subpixel grid of an LCD, the static shimmer of a cursor caught mid-blink. Each surface is meticulously crafted so that the eye can't decide whether it is looking at an icon or at an artifact photographed through a telephoto lens.
Composition is resolved through negative space. The frame is filled but never crowded; the focal element is given its own gravitational field. Axial and radial symmetries dominate because the subject is symmetric by nature — signal leaving and returning along the same line, one screen mirroring another. When symmetry is broken, it is broken precisely, by degrees measured in fractions of a point. The work should feel inevitable, as if the designer had no other choice. This is the hallmark of master-level execution — countless refinements hidden behind apparent ease.
Typography is absent. Labels are for product pages; the icon itself speaks only through form, color, and light. The mark must survive the brutal reduction of a Home Screen grid and still carry an idea. What remains after all explanation is stripped away is the whole point.
## The subtle reference
The Remote Framebuffer Protocol — the 1998 RFC on which VNC rests — describes a remote screen as a pixel rectangle updated over time. Every icon here is, at heart, a drawing of that rectangle. A rectangle seen through glass, a rectangle pointed at by a cursor, a rectangle broadcasting its state, a rectangle sliced by its own monogram, two rectangles tilted in conversation. The protocol is never named. But anyone who has spent long nights watching those updates arrive will feel its ghost.

BIN
icons/contact-sheet.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 326 KiB

628
icons/generate.py Normal file
View File

@@ -0,0 +1,628 @@
"""
Icon generator for Screens — 5 distinct 1024x1024 iOS app icons.
Philosophy: Luminous Distance (see PHILOSOPHY.md).
Each icon fills the canvas edge-to-edge. iOS applies the squircle mask.
"""
from __future__ import annotations
import math, os
from PIL import Image, ImageDraw, ImageFilter, ImageChops
SIZE = 1024
OUT = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- helpers
def blank(size=SIZE):
return Image.new("RGBA", (size, size), (0, 0, 0, 0))
def hex_to_rgba(h, a=255):
h = h.lstrip("#")
return (int(h[0:2], 16), int(h[2:4], 16), int(h[4:6], 16), a)
def lerp(a, b, t):
return a + (b - a) * t
def lerp_color(c1, c2, t):
return tuple(int(lerp(c1[i], c2[i], t)) for i in range(len(c1)))
def linear_gradient(size, stops, angle_deg=90):
"""stops = [(t, (r,g,b,a)), ...] with t in [0,1]; angle 0=→, 90=↓"""
w = h = size
img = Image.new("RGBA", (w, h))
px = img.load()
rad = math.radians(angle_deg)
dx, dy = math.cos(rad), math.sin(rad)
# Project (x,y) onto the direction vector, normalize to [0,1]
cx, cy = w / 2, h / 2
# half-extent in direction
ext = abs(dx) * w / 2 + abs(dy) * h / 2
for y in range(h):
for x in range(w):
t = ((x - cx) * dx + (y - cy) * dy) / (2 * ext) + 0.5
t = max(0.0, min(1.0, t))
# find segment
for i in range(len(stops) - 1):
t0, c0 = stops[i]
t1, c1 = stops[i + 1]
if t0 <= t <= t1:
local = (t - t0) / (t1 - t0) if t1 > t0 else 0
px[x, y] = lerp_color(c0, c1, local)
break
else:
px[x, y] = stops[-1][1]
return img
def radial_gradient(size, center, stops, radius=None):
w = h = size
if radius is None:
radius = size * 0.75
img = Image.new("RGBA", (w, h))
px = img.load()
cx, cy = center
for y in range(h):
for x in range(w):
dx, dy = x - cx, y - cy
d = math.sqrt(dx * dx + dy * dy) / radius
d = max(0.0, min(1.0, d))
for i in range(len(stops) - 1):
t0, c0 = stops[i]
t1, c1 = stops[i + 1]
if t0 <= d <= t1:
local = (d - t0) / (t1 - t0) if t1 > t0 else 0
px[x, y] = lerp_color(c0, c1, local)
break
else:
px[x, y] = stops[-1][1]
return img
def rounded_rect_mask(size, box, radius):
mask = Image.new("L", size, 0)
d = ImageDraw.Draw(mask)
d.rounded_rectangle(box, radius=radius, fill=255)
return mask
def drop_shadow(shape_mask, offset=(0, 12), blur=40, opacity=120):
sh = Image.new("RGBA", shape_mask.size, (0, 0, 0, 0))
sh.putalpha(shape_mask.point(lambda v: int(v * opacity / 255)))
sh = sh.filter(ImageFilter.GaussianBlur(blur))
offset_mask = Image.new("RGBA", shape_mask.size, (0, 0, 0, 0))
offset_mask.paste(sh, offset, sh)
return offset_mask
def specular_highlight(mask, intensity=110):
"""Top-inner glow hinting at glass."""
w, h = mask.size
grad = Image.new("L", (w, h))
for y in range(h):
v = int(255 * max(0, 1 - y / (h * 0.55)) ** 2 * (intensity / 255))
for x in range(w):
grad.putpixel((x, y), v)
highlight = Image.new("RGBA", (w, h), (255, 255, 255, 0))
highlight.putalpha(ImageChops.multiply(grad, mask))
return highlight
def paste_masked(base, layer, mask):
"""Paste `layer` onto `base` through `mask` (L mode)."""
combined = Image.new("RGBA", base.size, (0, 0, 0, 0))
combined.paste(layer, (0, 0), mask)
return Image.alpha_composite(base, combined)
def noise_layer(size, amount=6):
"""Fine film grain for tactile quality."""
import random
random.seed(42)
layer = Image.new("L", (size, size))
px = layer.load()
for y in range(size):
for x in range(size):
px[x, y] = 128 + random.randint(-amount, amount)
noise = Image.new("RGBA", (size, size), (255, 255, 255, 0))
noise.putalpha(layer.point(lambda v: abs(v - 128) * 2))
return noise
# ---------------------------------------------------------------- icon 1 — Portal
def icon_portal():
canvas = linear_gradient(SIZE, [
(0.0, hex_to_rgba("#0B0B2A")),
(0.6, hex_to_rgba("#1E1757")),
(1.0, hex_to_rgba("#2A0E4D")),
], angle_deg=115)
# Soft aura behind the portal
aura = radial_gradient(SIZE, (SIZE / 2, SIZE / 2 - 20), [
(0.0, hex_to_rgba("#7B6DF5", 230)),
(0.45, hex_to_rgba("#5B3FC0", 100)),
(1.0, hex_to_rgba("#140B36", 0)),
], radius=SIZE * 0.55)
canvas = Image.alpha_composite(canvas, aura)
# Screen rectangle
inset = 200
box = (inset, inset + 40, SIZE - inset, SIZE - inset - 40)
radius = 70
# Back-lit glow
glow_mask = rounded_rect_mask((SIZE, SIZE), (box[0] - 40, box[1] - 40, box[2] + 40, box[3] + 40), radius + 30)
glow = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
glow.putalpha(glow_mask.filter(ImageFilter.GaussianBlur(36)).point(lambda v: int(v * 0.85)))
glow_layer = Image.new("RGBA", (SIZE, SIZE), hex_to_rgba("#8A7BFF", 200))
glow_layer.putalpha(glow.getchannel("A"))
canvas = Image.alpha_composite(canvas, glow_layer)
# Screen surface gradient
screen = linear_gradient(SIZE, [
(0.0, hex_to_rgba("#CBB8FF")),
(0.35, hex_to_rgba("#7B63F0")),
(1.0, hex_to_rgba("#2B1C6B")),
], angle_deg=112)
screen_mask = rounded_rect_mask((SIZE, SIZE), box, radius)
canvas = paste_masked(canvas, screen, screen_mask)
# Inner bezel / rim light
rim_outer = rounded_rect_mask((SIZE, SIZE), box, radius)
inset_box = (box[0] + 8, box[1] + 8, box[2] - 8, box[3] - 8)
rim_inner = rounded_rect_mask((SIZE, SIZE), inset_box, radius - 8)
rim = ImageChops.subtract(rim_outer, rim_inner)
rim_layer = Image.new("RGBA", (SIZE, SIZE), hex_to_rgba("#E4D8FF", 150))
rim_layer.putalpha(rim)
canvas = Image.alpha_composite(canvas, rim_layer)
# Specular sheen (top-inner highlight, clipped to screen)
sheen = specular_highlight(screen_mask, intensity=90)
canvas = Image.alpha_composite(canvas, sheen)
# Inner scanline bloom — a soft horizontal ribbon implying content
ribbon_box = (box[0] + 80, int((box[1] + box[3]) / 2) - 3,
box[2] - 80, int((box[1] + box[3]) / 2) + 3)
ribbon_mask = rounded_rect_mask((SIZE, SIZE), ribbon_box, 3).filter(ImageFilter.GaussianBlur(4))
ribbon_layer = Image.new("RGBA", (SIZE, SIZE), hex_to_rgba("#FFFFFF", 180))
ribbon_layer.putalpha(ribbon_mask)
canvas = Image.alpha_composite(canvas, ribbon_layer)
canvas.save(os.path.join(OUT, "icon-1-portal.png"))
# ---------------------------------------------------------------- icon 2 — Cursor
def icon_cursor():
canvas = linear_gradient(SIZE, [
(0.0, hex_to_rgba("#2E36E8")),
(0.5, hex_to_rgba("#9B38E8")),
(1.0, hex_to_rgba("#FF5A8F")),
], angle_deg=135)
# Additional top-left glow to lift the cursor tip
glow = radial_gradient(SIZE, (SIZE * 0.32, SIZE * 0.30), [
(0.0, hex_to_rgba("#FFFFFF", 160)),
(0.5, hex_to_rgba("#B9B0FF", 60)),
(1.0, hex_to_rgba("#3A2C9A", 0)),
], radius=SIZE * 0.6)
canvas = Image.alpha_composite(canvas, glow)
# Classic macOS arrow cursor — authored as polygon coordinates on a 100-unit grid
cursor_norm = [
(0, 0), (0, 73), (20, 56), (32, 85), (44, 80), (33, 52), (56, 52)
]
# Scale + center the cursor
scale = 7.0
cursor_w = 56 * scale
cursor_h = 85 * scale
ox = (SIZE - cursor_w) / 2 - 20
oy = (SIZE - cursor_h) / 2 - 20
pts = [(ox + x * scale, oy + y * scale) for (x, y) in cursor_norm]
# Soft shadow behind cursor
shadow_mask = Image.new("L", (SIZE, SIZE), 0)
ImageDraw.Draw(shadow_mask).polygon([(p[0] + 16, p[1] + 24) for p in pts], fill=255)
shadow_mask = shadow_mask.filter(ImageFilter.GaussianBlur(22))
shadow_layer = Image.new("RGBA", (SIZE, SIZE), (20, 10, 60, 0))
shadow_layer.putalpha(shadow_mask.point(lambda v: int(v * 0.55)))
canvas = Image.alpha_composite(canvas, shadow_layer)
# Cursor outline (thin dark) and fill (white) — draw as two polygons
cursor_outer = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
od = ImageDraw.Draw(cursor_outer)
# Slight outline outset — simulate with stroked polygon
# Draw outline by expanding the polygon by ~stroke/2 via line draw
stroke = 18
od.polygon(pts, fill=(30, 14, 80, 255))
# Now shrink by drawing the white body slightly inset — simplest: re-use polygon, paint white with drawn stroke=0
cursor_body = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
bd = ImageDraw.Draw(cursor_body)
# Inset polygon: compute centroid and pull points toward it by `stroke*0.35` for subtle inset
cx = sum(p[0] for p in pts) / len(pts)
cy = sum(p[1] for p in pts) / len(pts)
inset = stroke * 0.35
inset_pts = []
for (x, y) in pts:
dx, dy = x - cx, y - cy
d = math.hypot(dx, dy)
if d == 0:
inset_pts.append((x, y))
else:
inset_pts.append((x - dx / d * inset, y - dy / d * inset))
bd.polygon(inset_pts, fill=(255, 255, 255, 255))
# Cursor highlight gradient over the body
body_mask = Image.new("L", (SIZE, SIZE), 0)
ImageDraw.Draw(body_mask).polygon(inset_pts, fill=255)
body_grad = linear_gradient(SIZE, [
(0.0, hex_to_rgba("#FFFFFF")),
(1.0, hex_to_rgba("#D8D4FF")),
], angle_deg=110)
cursor_body_grad = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
cursor_body_grad.paste(body_grad, (0, 0), body_mask)
canvas = Image.alpha_composite(canvas, cursor_outer)
canvas = Image.alpha_composite(canvas, cursor_body_grad)
canvas.save(os.path.join(OUT, "icon-2-cursor.png"))
# ---------------------------------------------------------------- icon 3 — Concentric Waves
def icon_waves():
canvas = linear_gradient(SIZE, [
(0.0, hex_to_rgba("#031028")),
(1.0, hex_to_rgba("#072047")),
], angle_deg=100)
# Deep radial dim at the edges for vignette
edge = radial_gradient(SIZE, (SIZE / 2, SIZE / 2), [
(0.0, hex_to_rgba("#000000", 0)),
(0.65, hex_to_rgba("#000000", 0)),
(1.0, hex_to_rgba("#000000", 120)),
], radius=SIZE * 0.75)
canvas = Image.alpha_composite(canvas, edge)
# Concentric rounded rectangles — 5 rings, smallest is bright, biggest is subtle
rings = [
{"inset": 120, "radius": 160, "stroke": 18, "color": hex_to_rgba("#1AA3C2", 110)},
{"inset": 210, "radius": 130, "stroke": 16, "color": hex_to_rgba("#1BC6E0", 150)},
{"inset": 300, "radius": 100, "stroke": 14, "color": hex_to_rgba("#31E1F5", 190)},
{"inset": 380, "radius": 74, "stroke": 12, "color": hex_to_rgba("#7EF0FF", 230)},
]
for r in rings:
box = (r["inset"], r["inset"], SIZE - r["inset"], SIZE - r["inset"])
outer = rounded_rect_mask((SIZE, SIZE), box, r["radius"])
inset_box = (box[0] + r["stroke"], box[1] + r["stroke"], box[2] - r["stroke"], box[3] - r["stroke"])
inner = rounded_rect_mask((SIZE, SIZE), inset_box, max(r["radius"] - r["stroke"], 10))
ring = ImageChops.subtract(outer, inner)
# Subtle blur — outer rings softer than inner
blur_amt = 0.8 + (rings.index(r)) * 0.3
ring = ring.filter(ImageFilter.GaussianBlur(blur_amt))
layer = Image.new("RGBA", (SIZE, SIZE), r["color"][:3] + (0,))
layer.putalpha(ring.point(lambda v, a=r["color"][3]: int(v * a / 255)))
canvas = Image.alpha_composite(canvas, layer)
# Central glowing dot
dot_r = 28
dot_box = (SIZE / 2 - dot_r, SIZE / 2 - dot_r, SIZE / 2 + dot_r, SIZE / 2 + dot_r)
dot_mask = Image.new("L", (SIZE, SIZE), 0)
ImageDraw.Draw(dot_mask).ellipse(dot_box, fill=255)
dot_mask = dot_mask.filter(ImageFilter.GaussianBlur(2))
dot_layer = Image.new("RGBA", (SIZE, SIZE), hex_to_rgba("#E8FDFF"))
dot_layer.putalpha(dot_mask)
canvas = Image.alpha_composite(canvas, dot_layer)
# Dot bloom
bloom_mask = Image.new("L", (SIZE, SIZE), 0)
ImageDraw.Draw(bloom_mask).ellipse((SIZE / 2 - 90, SIZE / 2 - 90, SIZE / 2 + 90, SIZE / 2 + 90), fill=255)
bloom_mask = bloom_mask.filter(ImageFilter.GaussianBlur(36))
bloom_layer = Image.new("RGBA", (SIZE, SIZE), hex_to_rgba("#7EF0FF"))
bloom_layer.putalpha(bloom_mask.point(lambda v: int(v * 0.45)))
canvas = Image.alpha_composite(canvas, bloom_layer)
canvas.save(os.path.join(OUT, "icon-3-waves.png"))
# ---------------------------------------------------------------- icon 4 — Monogram S
def icon_monogram():
canvas = linear_gradient(SIZE, [
(0.0, hex_to_rgba("#0A1430")),
(0.5, hex_to_rgba("#1B1450")),
(1.0, hex_to_rgba("#04061A")),
], angle_deg=125)
# Chromatic bloom — blue, magenta, amber — behind the mark
bloom1 = radial_gradient(SIZE, (SIZE * 0.28, SIZE * 0.30), [
(0.0, hex_to_rgba("#4AA8FF", 190)),
(1.0, hex_to_rgba("#080822", 0)),
], radius=SIZE * 0.55)
bloom2 = radial_gradient(SIZE, (SIZE * 0.78, SIZE * 0.78), [
(0.0, hex_to_rgba("#E845A8", 170)),
(1.0, hex_to_rgba("#080822", 0)),
], radius=SIZE * 0.55)
canvas = Image.alpha_composite(canvas, bloom1)
canvas = Image.alpha_composite(canvas, bloom2)
# Two interlocking chevrons that read as an "S"
# Upper chevron (like a `>` opening right) at top
# Lower chevron (like a `<` opening left) at bottom — stacked with overlap
stroke = 110
inset_x = 200
center_y = SIZE / 2
top_y = 260
bot_y = SIZE - 260
def chevron_points(start_x, end_x, tip_x, top, bottom, thickness):
"""A V-shaped chevron band from (start_x, top) → (tip_x, mid) → (end_x, top)
drawn as a parallelogram-like thick stroke."""
mid = (top + bottom) / 2
return [
(start_x, top),
(tip_x, bottom),
(end_x, top),
(end_x, top + thickness),
(tip_x, bottom + thickness), # overflowed — we'll draw differently
(start_x, top + thickness),
]
# Rather than manual polygons, paint two thick rounded "V" strokes.
upper = Image.new("L", (SIZE, SIZE), 0)
lower = Image.new("L", (SIZE, SIZE), 0)
ud = ImageDraw.Draw(upper)
ld = ImageDraw.Draw(lower)
# Upper chevron: start top-left → tip right-middle → continues back to top-right? Actually we want ">"
# We'll draw the S as two mirrored arcs — easier: use thick polylines along a cubic path.
# Approximate with chained line segments + round caps.
def thick_polyline(draw, pts, width):
for i in range(len(pts) - 1):
draw.line([pts[i], pts[i + 1]], fill=255, width=width)
for p in pts:
r = width / 2
draw.ellipse((p[0] - r, p[1] - r, p[0] + r, p[1] + r), fill=255)
# An abstract "S": top arc (left-top → right-center), bottom arc (right-center → left-bottom)
top_arc = [
(inset_x + 20, top_y),
(SIZE - inset_x - 80, top_y + 80),
(SIZE - inset_x, center_y - 40),
(SIZE - inset_x - 40, center_y),
]
bot_arc = [
(SIZE - inset_x - 40, center_y),
(inset_x + 40, center_y + 40),
(inset_x, bot_y - 80),
(inset_x + 20, bot_y),
]
# Smooth via many interpolated bezier-like points
def smooth(points, samples=80):
# Quadratic bezier through 4 points: treat as cubic bezier control poly
(p0, p1, p2, p3) = points
out = []
for i in range(samples + 1):
t = i / samples
# cubic bezier
x = (1 - t) ** 3 * p0[0] + 3 * (1 - t) ** 2 * t * p1[0] + 3 * (1 - t) * t ** 2 * p2[0] + t ** 3 * p3[0]
y = (1 - t) ** 3 * p0[1] + 3 * (1 - t) ** 2 * t * p1[1] + 3 * (1 - t) * t ** 2 * p2[1] + t ** 3 * p3[1]
out.append((x, y))
return out
top_curve = smooth(top_arc, samples=120)
bot_curve = smooth(bot_arc, samples=120)
thick_polyline(ud, top_curve, stroke)
thick_polyline(ld, bot_curve, stroke)
combined = ImageChops.lighter(upper, lower)
# Chromatic fill: horizontal-diagonal gradient inside the S
s_grad = linear_gradient(SIZE, [
(0.0, hex_to_rgba("#7FB2FF")),
(0.5, hex_to_rgba("#C97BFF")),
(1.0, hex_to_rgba("#FF8BB8")),
], angle_deg=120)
s_layer = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
s_layer.paste(s_grad, (0, 0), combined)
# Highlight rim on top edge of the S for glass feel
rim = combined.filter(ImageFilter.GaussianBlur(1))
rim_inner = combined.filter(ImageFilter.GaussianBlur(3))
edge = ImageChops.subtract(rim, rim_inner)
edge_layer = Image.new("RGBA", (SIZE, SIZE), hex_to_rgba("#FFFFFF", 130))
edge_layer.putalpha(edge)
# Drop shadow under the S
shadow = combined.filter(ImageFilter.GaussianBlur(26))
sh_layer = Image.new("RGBA", (SIZE, SIZE), (0, 0, 20, 0))
sh_layer.putalpha(shadow.point(lambda v: int(v * 0.55)))
sh_offset = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
sh_offset.paste(sh_layer, (0, 16), sh_layer)
canvas = Image.alpha_composite(canvas, sh_offset)
canvas = Image.alpha_composite(canvas, s_layer)
canvas = Image.alpha_composite(canvas, edge_layer)
canvas.save(os.path.join(OUT, "icon-4-monogram.png"))
# ---------------------------------------------------------------- icon 5 — Split Screen
def icon_split():
canvas = linear_gradient(SIZE, [
(0.0, hex_to_rgba("#1A1245")),
(0.5, hex_to_rgba("#2F1D78")),
(1.0, hex_to_rgba("#0D0824")),
], angle_deg=120)
# Warm highlight in upper-left
warm = radial_gradient(SIZE, (SIZE * 0.28, SIZE * 0.22), [
(0.0, hex_to_rgba("#F7B57A", 140)),
(1.0, hex_to_rgba("#1A1245", 0)),
], radius=SIZE * 0.55)
canvas = Image.alpha_composite(canvas, warm)
# Cool counterpoint lower-right
cool = radial_gradient(SIZE, (SIZE * 0.82, SIZE * 0.82), [
(0.0, hex_to_rgba("#4E6CFF", 180)),
(1.0, hex_to_rgba("#100828", 0)),
], radius=SIZE * 0.6)
canvas = Image.alpha_composite(canvas, cool)
# Two tilted rectangles — "back" one tilted left, "front" one tilted right.
def rotate_polygon(pts, angle_deg, cx, cy):
a = math.radians(angle_deg)
cos_a, sin_a = math.cos(a), math.sin(a)
out = []
for (x, y) in pts:
dx, dy = x - cx, y - cy
nx = dx * cos_a - dy * sin_a + cx
ny = dx * sin_a + dy * cos_a + cy
out.append((nx, ny))
return out
def draw_screen(rect_box, tilt_deg, body_grad_stops, rim_alpha=140):
# rect_box: (x0,y0,x1,y1)
x0, y0, x1, y1 = rect_box
cx, cy = (x0 + x1) / 2, (y0 + y1) / 2
# Approximate rounded-rect by drawing on a tilted transparent layer
layer = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
mask = Image.new("L", (SIZE, SIZE), 0)
radius = 80
# Draw axis-aligned rounded rect on a working canvas sized to the screen's bounding box, then rotate.
local_w = int(x1 - x0) + 200
local_h = int(y1 - y0) + 200
local_mask = Image.new("L", (local_w, local_h), 0)
lx0 = 100
ly0 = 100
lx1 = local_w - 100
ly1 = local_h - 100
ImageDraw.Draw(local_mask).rounded_rectangle((lx0, ly0, lx1, ly1), radius=radius, fill=255)
local_mask = local_mask.rotate(tilt_deg, resample=Image.BICUBIC, expand=False)
# Place local mask into full-size mask at correct position
place_x = int(cx - local_w / 2)
place_y = int(cy - local_h / 2)
mask.paste(local_mask, (place_x, place_y))
# Body gradient
body = linear_gradient(SIZE, body_grad_stops, angle_deg=110)
# Shadow before composite
shadow_mask = mask.filter(ImageFilter.GaussianBlur(30))
shadow = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
shadow.putalpha(shadow_mask.point(lambda v: int(v * 0.55)))
sh_offset = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
sh_offset.paste(shadow, (14, 30), shadow)
body_layer = Image.new("RGBA", (SIZE, SIZE), (0, 0, 0, 0))
body_layer.paste(body, (0, 0), mask)
# Rim / edge highlight
rim_outer = mask
rim_inner = mask.filter(ImageFilter.GaussianBlur(2)).point(lambda v: 255 if v > 180 else 0)
rim_inner_eroded = rim_inner.filter(ImageFilter.MinFilter(9))
rim = ImageChops.subtract(rim_outer, rim_inner_eroded)
rim_layer = Image.new("RGBA", (SIZE, SIZE), hex_to_rgba("#FFFFFF", rim_alpha))
rim_layer.putalpha(rim)
return sh_offset, body_layer, rim_layer, mask
# Back screen: bigger, tilted left
back_box = (200, 250, SIZE - 240, SIZE - 200)
back = draw_screen(
back_box, tilt_deg=-7,
body_grad_stops=[
(0.0, hex_to_rgba("#5A4FD9")),
(1.0, hex_to_rgba("#1E154F")),
],
rim_alpha=90
)
# Front screen: smaller, tilted right, shifted down-right
front_box = (280, 330, SIZE - 160, SIZE - 160)
front = draw_screen(
front_box, tilt_deg=6,
body_grad_stops=[
(0.0, hex_to_rgba("#F5CE9A")),
(0.5, hex_to_rgba("#E086A3")),
(1.0, hex_to_rgba("#6B488C")),
],
rim_alpha=160
)
# Composite: back shadow → back body → back rim → front shadow → front body → front rim
for bundle in (back, front):
sh, body, rim, _ = bundle
canvas = Image.alpha_composite(canvas, sh)
canvas = Image.alpha_composite(canvas, body)
canvas = Image.alpha_composite(canvas, rim)
# Specular highlight on front screen
_, _, _, front_mask = front
sheen = specular_highlight(front_mask, intensity=120)
canvas = Image.alpha_composite(canvas, sheen)
canvas.save(os.path.join(OUT, "icon-5-split.png"))
# ---------------------------------------------------------------- contact sheet
def contact_sheet():
paths = [
"icon-1-portal.png",
"icon-2-cursor.png",
"icon-3-waves.png",
"icon-4-monogram.png",
"icon-5-split.png",
]
cell = 360
gap = 30
cols = 5
rows = 1
label_h = 70
W = cols * cell + (cols + 1) * gap
H = rows * cell + (rows + 1) * gap + label_h
sheet = Image.new("RGB", (W, H), (24, 24, 30))
d = ImageDraw.Draw(sheet)
for i, p in enumerate(paths):
img = Image.open(os.path.join(OUT, p)).convert("RGBA")
# Apply iOS squircle preview mask
mask = Image.new("L", img.size, 0)
ImageDraw.Draw(mask).rounded_rectangle((0, 0, img.size[0], img.size[1]), radius=int(img.size[0] * 0.22), fill=255)
img.putalpha(mask)
img = img.resize((cell, cell), Image.LANCZOS)
x = gap + i * (cell + gap)
y = gap
sheet.paste(img, (x, y), img)
label = ["1 Portal", "2 Cursor", "3 Waves", "4 Monogram", "5 Split"][i]
d.text((x + cell / 2 - 30, y + cell + 20), label, fill=(240, 240, 250))
sheet.save(os.path.join(OUT, "contact-sheet.png"))
if __name__ == "__main__":
print("Generating icons…")
icon_portal()
print(" ✓ portal")
icon_cursor()
print(" ✓ cursor")
icon_waves()
print(" ✓ waves")
icon_monogram()
print(" ✓ monogram")
icon_split()
print(" ✓ split")
contact_sheet()
print(" ✓ contact sheet")
print("Done.")

BIN
icons/icon-1-portal.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 200 KiB

BIN
icons/icon-2-cursor.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 244 KiB

BIN
icons/icon-3-waves.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 121 KiB

BIN
icons/icon-4-monogram.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 288 KiB

BIN
icons/icon-5-split.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 249 KiB