Phase 4-5: build pipeline + device enrollment
Builder service (Mac mini): - Build worker: xcodebuild archive + export + fastlane signing + upload to unraid - /api/build/upload (source archive) and /api/build/git (clone) ingest paths - SSE-streamed build logs, builds list UI, live status updates - /api/devices/from-enrollment bridge endpoint (shared-secret auth) Storefront (unraid): - /enroll/ public flow: landing page, mobileconfig generator, callback parser - Forwards extracted UDIDs to the Mac mini builder for ASC registration - docker-compose.yml now passes BUILDER_URL and BUILDER_SHARED_SECRET Updated CLAUDE.md with full architecture, deploy flow, and gotchas.
This commit is contained in:
@@ -12,3 +12,7 @@ BASE_URL=https://appstore.example.com
|
||||
|
||||
# Port
|
||||
PORT=3000
|
||||
|
||||
# Mac mini builder bridge (used by /enroll/callback to forward UDIDs)
|
||||
BUILDER_URL=http://10.3.3.192:3090
|
||||
BUILDER_SHARED_SECRET=changeme-same-as-builder-env
|
||||
|
||||
196
CLAUDE.md
196
CLAUDE.md
@@ -1,79 +1,191 @@
|
||||
# iOS App Store
|
||||
|
||||
Self-hosted iOS OTA distribution server. Node.js/Express + SQLite + Docker, deployed on unraid behind Nginx Proxy Manager.
|
||||
Hybrid iOS distribution system. Two independent services sharing one git repo:
|
||||
|
||||
- **`/` (root)** — the public **storefront** on unraid. Hosts IPAs, serves OTA installs, handles device enrollment. Reached by iPhones at `https://appstore.treytartt.com`.
|
||||
- **`builder/`** — the private **build console** on the Mac mini. Takes source code (archive or git URL), runs xcodebuild + fastlane, pushes finished IPAs to the storefront. Reached by the developer on the LAN at `http://Treys-Mac-mini.local:3090` (`10.3.3.192:3090`).
|
||||
|
||||
The split exists because `xcodebuild` needs macOS and the Mac mini is the only mac we have, but unraid is a much better place to store + serve IPAs long-term.
|
||||
|
||||
## Live deployment
|
||||
|
||||
- **URL**: https://appstore.treytartt.com
|
||||
- **Container**: `ios-appstore` on unraid (port `3080` internally, proxied via NPM at host `10.3.3.11:3080`)
|
||||
- **App code**: `/mnt/user/appdata/ios-appstore/` on unraid (Dockerfile, source, compose, `.env`)
|
||||
- **Data volume**: `/mnt/user/downloads/ios-appstore/` mounted as `/data` (SQLite DB, IPAs, icons)
|
||||
### Storefront (unraid)
|
||||
|
||||
This split is intentional — app code in appdata, persistent data in downloads. Don't put data volumes in appdata or app source in downloads.
|
||||
- **URL**: https://appstore.treytartt.com (public, Let's Encrypt via NPM)
|
||||
- **Container**: `ios-appstore` on unraid, port `3080` internally
|
||||
- **App code**: `/mnt/user/appdata/ios-appstore/` (Dockerfile, source, compose, `.env`)
|
||||
- **Data volume**: `/mnt/user/downloads/ios-appstore/` → mounted as `/data` (SQLite DB, IPAs, icons)
|
||||
- **NPM proxy host #16**: `appstore.treytartt.com` → `10.3.3.11:3080`, SSL forced
|
||||
- **Env vars** (in `.env` on the server):
|
||||
- `ADMIN_PASSWORD`, `API_TOKEN`, `SESSION_SECRET`, `BASE_URL`
|
||||
- `BUILDER_URL=http://10.3.3.192:3090` — LAN address of the Mac mini builder
|
||||
- `BUILDER_SHARED_SECRET` — must match `builder/.env` on the Mac mini
|
||||
|
||||
The `.env` on the server holds `ADMIN_PASSWORD`, `API_TOKEN`, and `SESSION_SECRET`. Read it from `/mnt/user/appdata/ios-appstore/.env` when you need them.
|
||||
### Builder (Mac mini)
|
||||
|
||||
- **URL**: http://Treys-Mac-mini.local:3090 (LAN-only, no SSL, no public DNS)
|
||||
- **Native Node** (not Docker — `xcodebuild` can't run in a container on macOS)
|
||||
- **App code**: `/Users/m4mini/AppStoreBuilder/app/` (copied from `builder/` subtree via `builder/bin/deploy.sh`)
|
||||
- **Data**: `/Users/m4mini/AppStoreBuilder/data/` (SQLite + ASC keys + source archives + build artifacts + logs)
|
||||
- **Process supervision**: launchd — `~/Library/LaunchAgents/com.88oak.appstorebuilder.plist` (KeepAlive, RunAtLoad)
|
||||
- **Env vars** (in `builder/.env`, loaded non-destructively by `src/server.js`):
|
||||
- `ADMIN_PASSWORD`, `SESSION_SECRET`, `DATA_DIR`, `PORT`, `BUILDER_SHARED_SECRET`
|
||||
|
||||
**Important**: The builder code must NOT live under `~/Desktop/` when running via launchd. macOS TCC blocks launchd-spawned processes from reading Desktop, which causes the Node process to hang on `__getcwd` during startup. That's why we copy to `/Users/m4mini/AppStoreBuilder/app/` via the deploy script instead of pointing launchd directly at the git checkout in `~/Desktop/code/ios-appstore/builder/`.
|
||||
|
||||
## Deploy flow
|
||||
|
||||
### Storefront (unraid)
|
||||
|
||||
```bash
|
||||
# 1. Sync local changes to unraid (excludes node_modules, data, .env)
|
||||
rsync -avz --exclude node_modules --exclude data --exclude .env \
|
||||
# Sync changes (excluding builder/, data, .env, node_modules)
|
||||
rsync -avz --exclude node_modules --exclude data --exclude .env --exclude builder \
|
||||
/Users/m4mini/Desktop/code/ios-appstore/ \
|
||||
unraid:/mnt/user/appdata/ios-appstore/
|
||||
|
||||
# 2. Rebuild and restart
|
||||
# Rebuild + restart
|
||||
ssh unraid "cd /mnt/user/appdata/ios-appstore && docker compose up -d --build"
|
||||
|
||||
# 3. Verify
|
||||
ssh unraid "docker logs ios-appstore --tail 20 && curl -s http://localhost:3080/api/health"
|
||||
# Verify
|
||||
curl -s https://appstore.treytartt.com/api/health
|
||||
```
|
||||
|
||||
`docker-compose.yml` builds the image from local source — no registry. The data volume persists across rebuilds.
|
||||
### Builder (Mac mini)
|
||||
|
||||
## Architecture
|
||||
```bash
|
||||
# Single command: rsync to /Users/m4mini/AppStoreBuilder/app/, kickstart launchd, health check
|
||||
/Users/m4mini/Desktop/code/ios-appstore/builder/bin/deploy.sh
|
||||
```
|
||||
|
||||
- `src/server.js` — Express app, all routes, multer upload handling
|
||||
## Architecture overview
|
||||
|
||||
```
|
||||
iPhone ────► https://appstore.treytartt.com ◄──── Developer on LAN
|
||||
│ │
|
||||
│ │
|
||||
▼ ▼
|
||||
NPM + LE (unraid) http://Treys-Mac-mini.local:3090
|
||||
│ │
|
||||
▼ ▼
|
||||
ios-appstore container AppStoreBuilder (Node + launchd)
|
||||
(Docker, Node 20 Alpine) ─────────────────────────────────
|
||||
───────────────────── • Source upload + git clone
|
||||
• /api/apps browse • xcodebuild archive + export
|
||||
• /api/upload (IPAs in) • fastlane sigh (ad-hoc profiles)
|
||||
• /api/manifest OTA • ASC API (devices, profiles)
|
||||
• /api/download IPA • Build queue + log streaming
|
||||
• /enroll/ public flow ─────►• /api/devices/from-enrollment
|
||||
▲ │
|
||||
│ │
|
||||
└──────────────────────────────┘
|
||||
finished IPAs POSTed
|
||||
to /api/upload via
|
||||
the existing API_TOKEN
|
||||
```
|
||||
|
||||
## Storefront (root `/src/`)
|
||||
|
||||
- `src/server.js` — Express app, all routes, multer upload handling, `/enroll/*` bridge
|
||||
- `src/db.js` — SQLite schema (apps, builds, devices)
|
||||
- `src/ipa-parser.js` — Unzips IPA, extracts `Info.plist` and app icon
|
||||
- `src/manifest.js` — Generates the OTA manifest plist iOS fetches
|
||||
- `src/mobileconfig.js` — Generates the `.mobileconfig` Profile Service payload and parses callback plists
|
||||
- `src/auth.js` — Session middleware (web UI) + token middleware (API)
|
||||
- `views/` — Login, app listing, upload pages (vanilla HTML)
|
||||
- `views/` — Login, app listing, upload, enroll pages
|
||||
- `public/` — CSS + client-side JS
|
||||
|
||||
Apps are keyed by bundle ID. Uploading the same bundle ID adds a new build to the existing app instead of duplicating it.
|
||||
### Storefront auth
|
||||
|
||||
## Auth model
|
||||
- **Session cookies** for browser users (`ADMIN_PASSWORD`)
|
||||
- **`X-Api-Token` header** for CLI/automation (`API_TOKEN`) — this is what the Mac mini uses when POSTing IPAs
|
||||
- **Public (no auth)**: `/api/manifest/:id`, `/api/download/:id`, `/enroll/*` — iOS fetches these unauthenticated
|
||||
|
||||
Two parallel auth schemes:
|
||||
- **Session cookies** for browser users (login at `/login` with `ADMIN_PASSWORD`)
|
||||
- **`X-Api-Token` header** for CLI/automation (`API_TOKEN`)
|
||||
## Builder (`builder/`)
|
||||
|
||||
`requireAuth` accepts either. The manifest and IPA download endpoints (`/api/manifest/:id`, `/api/download/:id`) are intentionally **public** — iOS fetches them unauthenticated during the OTA install.
|
||||
- `builder/src/server.js` — Express app, session auth, mounts all routes, starts the build worker
|
||||
- `builder/src/db.js` — SQLite schema (settings, devices, apps, profiles, build_jobs)
|
||||
- `builder/src/auth.js` — Session (web UI) + shared-secret (enrollment bridge from unraid)
|
||||
- `builder/src/asc-api.js` — App Store Connect REST client (ES256 JWT, `/v1/devices`, `/v1/profiles`, `/v1/bundleIds`)
|
||||
- `builder/src/profile-manager.js` — Wraps `fastlane sigh`, caches `.mobileprovision` files, auto-installs into `~/Library/MobileDevice/Provisioning Profiles/`
|
||||
- `builder/src/build-worker.js` — In-process build queue: preparing → signing → archiving → exporting → uploading → succeeded
|
||||
- `builder/src/build-routes.js` — `/api/build/upload`, `/api/build/git`, `/api/builds`, `/api/builds/:id/logs` (SSE)
|
||||
- `builder/fastlane/Fastfile` — Single `generate_adhoc` lane using the ASC API key
|
||||
- `builder/bin/deploy.sh` — Copies source to `/Users/m4mini/AppStoreBuilder/app/` and kickstarts launchd
|
||||
- `builder/views/` — Login, builds, build, devices, settings pages
|
||||
- `builder/public/` — CSS (copied from the storefront for visual continuity) + client JS
|
||||
|
||||
## OTA gotchas
|
||||
### How a build works
|
||||
|
||||
- **Development-signed IPAs cannot be installed OTA.** iOS only allows OTA installs of `ad-hoc` or `enterprise`-signed builds. If a user reports "integrity could not be verified", first check the export method in their `ExportOptions.plist`.
|
||||
- Ad-hoc requires a **distribution certificate** (not a development cert) and an **ad-hoc provisioning profile** with the target device UDIDs registered.
|
||||
- HTTPS is mandatory and must use a trusted CA. Self-signed certs don't work on iOS 12+. NPM's Let's Encrypt cert handles this.
|
||||
- The manifest's `bundle-identifier` must exactly match the IPA's bundle ID.
|
||||
1. User posts source (.zip/.tar.gz or git URL) to `/api/build/upload` or `/api/build/git`. Server creates a `build_jobs` row with `status=pending`, extracts/clones the source into `data/source/<job-id>/`, and kicks the worker.
|
||||
2. **preparing**: worker finds `.xcodeproj`/`.xcworkspace`, picks a scheme, runs `xcodebuild -showBuildSettings -json` to extract every target's `PRODUCT_BUNDLE_IDENTIFIER` and the `DEVELOPMENT_TEAM`.
|
||||
3. **signing**: for each bundle ID, `profile-manager.getProfile()` ensures a fresh ad-hoc profile exists — serving from cache if possible, running `fastlane sigh` if stale. Each profile is installed into `~/Library/MobileDevice/Provisioning Profiles/` so `xcodebuild` finds it.
|
||||
4. **archiving**: `xcodebuild archive` with `CODE_SIGN_STYLE=Manual`, the detected team ID, `-allowProvisioningUpdates`. xcodebuild matches bundle IDs to the pre-installed profiles automatically.
|
||||
5. **exporting**: generate `ExportOptions.plist` with `method=ad-hoc` and the full `provisioningProfiles` map, then `xcodebuild -exportArchive`.
|
||||
6. **uploading**: the produced `.ipa` is POSTed to `https://appstore.treytartt.com/api/upload` using the existing `API_TOKEN`. The storefront's existing parser + DB insert + manifest generation run unchanged.
|
||||
7. **succeeded**: clean up source + archive (keep log + IPA + ExportOptions.plist in `data/build/<job-id>/`).
|
||||
|
||||
## Testing changes
|
||||
Profile cache invalidation: whenever a device is added/deleted (manually via UI or via the enrollment bridge), `invalidateProfilesForDeviceChange()` clears `updated_at` on every row in `profiles`, forcing the next build to regenerate via `sigh`.
|
||||
|
||||
For backend changes, after deploying:
|
||||
```bash
|
||||
# Health
|
||||
curl -s https://appstore.treytartt.com/api/health
|
||||
## Enrollment flow
|
||||
|
||||
# List apps (needs token)
|
||||
curl -s -H "X-Api-Token: $TOKEN" https://appstore.treytartt.com/api/apps
|
||||
1. Tester opens `https://appstore.treytartt.com/enroll` on their iPhone.
|
||||
2. Tap "Install Profile" → downloads `/enroll/profile.mobileconfig` (Profile Service payload pointing back at `/enroll/callback`). iOS shows "Not Signed" — acceptable for an internal store, just tap Install.
|
||||
3. iOS installs the profile, collects device attributes (UDID, PRODUCT, VERSION, DEVICE_NAME, SERIAL), wraps them in a CMS-signed plist, and POSTs to `/enroll/callback`.
|
||||
4. The unraid storefront doesn't validate the CMS signature — it just scans the raw body for the inner `<?xml … </plist>` block and extracts UDID/name/model.
|
||||
5. The unraid server forwards `{udid, name, model}` to the Mac mini at `http://10.3.3.192:3090/api/devices/from-enrollment` with `Authorization: Bearer <BUILDER_SHARED_SECRET>`.
|
||||
6. The builder's `/api/devices/from-enrollment` endpoint:
|
||||
- Upserts into the local `devices` table
|
||||
- Calls `asc.registerDevice()` to register with Apple via the App Store Connect API
|
||||
- Clears the profile cache so the next build picks up the new device
|
||||
7. iOS is redirected (303) to `/enroll/success`.
|
||||
|
||||
# Upload an IPA
|
||||
curl -X POST https://appstore.treytartt.com/api/upload \
|
||||
-H "X-Api-Token: $TOKEN" \
|
||||
-F "ipa=@/path/to/App.ipa" \
|
||||
-F "notes=test"
|
||||
```
|
||||
The shared secret lives in both `.env` files and must match. Rotate by updating both sides.
|
||||
|
||||
Get `$TOKEN` from `/mnt/user/appdata/ios-appstore/.env` on unraid.
|
||||
## Endpoints quick reference
|
||||
|
||||
For frontend changes, just rsync + restart and refresh the browser.
|
||||
### Storefront (public URL)
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/` | session | App listing UI |
|
||||
| GET | `/upload` | session | Upload page |
|
||||
| POST | `/api/upload` | token | Upload IPA (used by Mac mini builder) |
|
||||
| GET | `/api/apps` | token | List apps |
|
||||
| GET | `/api/manifest/:id` | public | iOS install manifest |
|
||||
| GET | `/api/download/:id` | public | Download IPA |
|
||||
| GET | `/enroll` | public | Enrollment landing page |
|
||||
| GET | `/enroll/profile.mobileconfig` | public | Profile Service payload |
|
||||
| POST | `/enroll/callback` | public (raw body) | iOS UDID posts here |
|
||||
| GET | `/enroll/success` | public | Post-enrollment page |
|
||||
|
||||
### Builder (LAN URL)
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/` | session | Build list UI |
|
||||
| GET | `/build` | session | New build form |
|
||||
| GET | `/devices` | session | Device management UI |
|
||||
| GET | `/settings` | session | ASC + unraid settings |
|
||||
| POST | `/api/build/upload` | session | Upload source archive |
|
||||
| POST | `/api/build/git` | session | Clone git repo |
|
||||
| GET | `/api/builds` | session | List jobs |
|
||||
| GET | `/api/builds/:id/logs` | session | SSE log stream |
|
||||
| GET | `/api/devices` | session | List devices |
|
||||
| POST | `/api/devices` | session | Register (local + ASC) |
|
||||
| POST | `/api/devices/from-enrollment` | shared secret | Called by unraid enroll bridge |
|
||||
| GET | `/api/profile/:bundleId` | session | Fetch/generate ad-hoc profile |
|
||||
| POST | `/api/settings/test-asc` | session | Verify ASC key works |
|
||||
| POST | `/api/settings/test-unraid` | session | Verify unraid API token works |
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Dev-signed IPAs cannot be installed OTA.** The builder always produces ad-hoc builds via fastlane sigh. Manual uploads via `/api/upload` still work, but the uploader is responsible for the signing method.
|
||||
- **`builder/` code must not live under `~/Desktop/`** when launchd runs it — TCC blocks the process. That's why `deploy.sh` copies to `/Users/m4mini/AppStoreBuilder/app/`.
|
||||
- **Docker compose env vars are substituted at compose-time** from the host `.env`, not read from `.env` at runtime inside the container. When adding a new env var to the storefront, update BOTH `.env` on the server AND `docker-compose.yml`.
|
||||
- **The builder's Node version is 25.x**, the storefront's container is Node 20. They're independent; mixing package-lock.json files between the two will not work.
|
||||
- **Device cache invalidation is eager**: adding or removing any device marks every profile stale. This is fine for a personal setup but would be worth scoping per-app in a larger deployment.
|
||||
|
||||
## Local credentials (look these up, don't hardcode)
|
||||
|
||||
- Storefront admin password, API token, session secret: `/mnt/user/appdata/ios-appstore/.env` on unraid
|
||||
- Builder admin password, session secret, shared secret: `/Users/m4mini/Desktop/code/ios-appstore/builder/.env` (dev) and `/Users/m4mini/AppStoreBuilder/app/.env` (deployed — same file, synced via deploy.sh excluding .env so they can diverge)
|
||||
|
||||
51
builder/public/js/build.js
Normal file
51
builder/public/js/build.js
Normal file
@@ -0,0 +1,51 @@
|
||||
const $ = (s) => document.querySelector(s);
|
||||
function toast(msg, kind = '') {
|
||||
const t = $('#toast');
|
||||
t.textContent = msg;
|
||||
t.className = 'toast show ' + kind;
|
||||
setTimeout(() => t.classList.remove('show'), 3500);
|
||||
}
|
||||
|
||||
$('#upload-form').addEventListener('submit', async (e) => {
|
||||
e.preventDefault();
|
||||
const fd = new FormData(e.target);
|
||||
const btn = e.target.querySelector('button[type=submit]');
|
||||
btn.disabled = true;
|
||||
btn.textContent = 'Uploading…';
|
||||
try {
|
||||
const r = await fetch('/api/build/upload', { method: 'POST', body: fd });
|
||||
const data = await r.json();
|
||||
if (!r.ok) throw new Error(data.error || 'Upload failed');
|
||||
location.href = `/builds#${data.job_id}`;
|
||||
} catch (err) {
|
||||
toast(err.message, 'error');
|
||||
btn.disabled = false;
|
||||
btn.textContent = 'Queue Build';
|
||||
}
|
||||
});
|
||||
|
||||
$('#git-form').addEventListener('submit', async (e) => {
|
||||
e.preventDefault();
|
||||
const body = {
|
||||
url: e.target.url.value.trim(),
|
||||
branch: e.target.branch.value.trim() || null,
|
||||
scheme: e.target.scheme.value.trim() || null,
|
||||
};
|
||||
const btn = e.target.querySelector('button[type=submit]');
|
||||
btn.disabled = true;
|
||||
btn.textContent = 'Cloning…';
|
||||
try {
|
||||
const r = await fetch('/api/build/git', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
const data = await r.json();
|
||||
if (!r.ok) throw new Error(data.error || 'Clone failed');
|
||||
location.href = `/builds#${data.job_id}`;
|
||||
} catch (err) {
|
||||
toast(err.message, 'error');
|
||||
btn.disabled = false;
|
||||
btn.textContent = 'Queue Build';
|
||||
}
|
||||
});
|
||||
107
builder/public/js/builds.js
Normal file
107
builder/public/js/builds.js
Normal file
@@ -0,0 +1,107 @@
|
||||
const $ = (s) => document.querySelector(s);
|
||||
const esc = (s) => { const d = document.createElement('div'); d.textContent = s ?? ''; return d.innerHTML; };
|
||||
|
||||
let currentEventSource = null;
|
||||
|
||||
function formatDate(s) {
|
||||
if (!s) return '—';
|
||||
return new Date(s + 'Z').toLocaleString();
|
||||
}
|
||||
|
||||
function duration(start, end) {
|
||||
if (!start) return '—';
|
||||
const s = new Date(start + 'Z').getTime();
|
||||
const e = end ? new Date(end + 'Z').getTime() : Date.now();
|
||||
const secs = Math.max(0, Math.round((e - s) / 1000));
|
||||
if (secs < 60) return `${secs}s`;
|
||||
return `${Math.floor(secs / 60)}m ${secs % 60}s`;
|
||||
}
|
||||
|
||||
function statusBadge(status) {
|
||||
const running = ['preparing', 'signing', 'archiving', 'exporting', 'uploading'].includes(status);
|
||||
const cls = status === 'succeeded' ? 'succeeded'
|
||||
: status === 'failed' ? 'failed'
|
||||
: running ? 'running'
|
||||
: 'pending';
|
||||
return `<span class="badge ${cls}">${status}</span>`;
|
||||
}
|
||||
|
||||
async function loadJobs() {
|
||||
const r = await fetch('/api/builds');
|
||||
if (r.status === 401) { location.href = '/login'; return; }
|
||||
const jobs = await r.json();
|
||||
const container = $('#jobs-container');
|
||||
|
||||
if (!jobs.length) {
|
||||
container.innerHTML = '<div class="card"><p style="color:var(--text-muted)">No builds yet. Start one from <a href="/build" style="color:var(--accent)">New Build</a>.</p></div>';
|
||||
return;
|
||||
}
|
||||
|
||||
container.innerHTML = `
|
||||
<table class="table">
|
||||
<thead>
|
||||
<tr><th>Status</th><th>Bundle</th><th>Source</th><th>Started</th><th>Duration</th><th></th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
${jobs.map(j => `
|
||||
<tr data-id="${j.id}" style="cursor:pointer">
|
||||
<td>${statusBadge(j.status)}</td>
|
||||
<td class="mono">${esc(j.bundle_id) || '<span style="color:var(--text-muted)">—</span>'}</td>
|
||||
<td class="mono">${esc(j.source_kind)}: ${esc((j.source_ref || '').slice(0, 40))}</td>
|
||||
<td class="mono">${esc(formatDate(j.started_at))}</td>
|
||||
<td class="mono">${esc(duration(j.started_at, j.finished_at))}</td>
|
||||
<td>${j.install_url ? `<a href="${esc(j.install_url)}" class="btn-sm" style="background:var(--accent);color:white;padding:5px 12px;border-radius:14px;text-decoration:none;font-size:12px">Install</a>` : ''}</td>
|
||||
</tr>
|
||||
`).join('')}
|
||||
</tbody>
|
||||
</table>
|
||||
`;
|
||||
|
||||
container.querySelectorAll('tbody tr').forEach((tr) => {
|
||||
tr.addEventListener('click', () => openJob(tr.getAttribute('data-id')));
|
||||
});
|
||||
}
|
||||
|
||||
async function openJob(id) {
|
||||
location.hash = id;
|
||||
const r = await fetch(`/api/builds/${id}`);
|
||||
if (!r.ok) return;
|
||||
const job = await r.json();
|
||||
$('#detail').style.display = 'block';
|
||||
$('#detail-title').textContent = `Job ${id.slice(0, 8)} · ${job.status}`;
|
||||
$('#detail-meta').innerHTML = `
|
||||
<div>bundle: ${esc(job.bundle_id || '—')}</div>
|
||||
<div>scheme: ${esc(job.scheme || '—')}</div>
|
||||
<div>source: ${esc(job.source_kind)} ${esc(job.source_ref || '')}</div>
|
||||
<div>started: ${esc(formatDate(job.started_at))} · finished: ${esc(formatDate(job.finished_at))}</div>
|
||||
${job.install_url ? `<div>install: <a href="${esc(job.install_url)}" style="color:var(--accent)">${esc(job.install_url.slice(0, 80))}…</a></div>` : ''}
|
||||
${job.error ? `<div style="color:var(--danger)">error: ${esc(job.error)}</div>` : ''}
|
||||
`;
|
||||
|
||||
const logEl = $('#log-viewer');
|
||||
logEl.textContent = '';
|
||||
|
||||
if (currentEventSource) { currentEventSource.close(); currentEventSource = null; }
|
||||
|
||||
const es = new EventSource(`/api/builds/${id}/logs`);
|
||||
currentEventSource = es;
|
||||
es.onmessage = (ev) => {
|
||||
logEl.textContent += ev.data + '\n';
|
||||
logEl.scrollTop = logEl.scrollHeight;
|
||||
};
|
||||
es.addEventListener('done', () => {
|
||||
es.close();
|
||||
currentEventSource = null;
|
||||
// Refresh the job list so status pill updates.
|
||||
loadJobs();
|
||||
});
|
||||
es.onerror = () => { es.close(); };
|
||||
}
|
||||
|
||||
loadJobs();
|
||||
setInterval(loadJobs, 5000);
|
||||
|
||||
// If arriving with a hash, open that job.
|
||||
if (location.hash.length > 1) {
|
||||
openJob(location.hash.slice(1));
|
||||
}
|
||||
199
builder/src/build-routes.js
Normal file
199
builder/src/build-routes.js
Normal file
@@ -0,0 +1,199 @@
|
||||
// Build pipeline HTTP routes.
|
||||
// Attached to the main Express app in server.js via `register(app)`.
|
||||
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const os = require('os');
|
||||
const { spawn } = require('child_process');
|
||||
const multer = require('multer');
|
||||
const { v4: uuidv4 } = require('uuid');
|
||||
|
||||
const { db, DATA_DIR } = require('./db');
|
||||
const buildWorker = require('./build-worker');
|
||||
|
||||
const SOURCE_DIR = path.join(DATA_DIR, 'source');
|
||||
const LOGS_DIR = path.join(DATA_DIR, 'builds');
|
||||
const TMP_DIR = path.join(DATA_DIR, 'tmp');
|
||||
|
||||
[SOURCE_DIR, LOGS_DIR, TMP_DIR].forEach((d) => fs.mkdirSync(d, { recursive: true }));
|
||||
|
||||
const archiveUpload = multer({
|
||||
dest: TMP_DIR,
|
||||
limits: { fileSize: 500 * 1024 * 1024 },
|
||||
fileFilter: (req, file, cb) => {
|
||||
const name = file.originalname.toLowerCase();
|
||||
if (name.endsWith('.zip') || name.endsWith('.tar.gz') || name.endsWith('.tgz')) {
|
||||
return cb(null, true);
|
||||
}
|
||||
cb(new Error('Only .zip, .tar.gz, or .tgz archives'));
|
||||
},
|
||||
});
|
||||
|
||||
function extractArchive(archivePath, destDir) {
|
||||
return new Promise((resolve, reject) => {
|
||||
fs.mkdirSync(destDir, { recursive: true });
|
||||
const lower = archivePath.toLowerCase();
|
||||
let cmd, args;
|
||||
if (lower.endsWith('.zip')) {
|
||||
cmd = '/usr/bin/unzip';
|
||||
args = ['-q', archivePath, '-d', destDir];
|
||||
} else {
|
||||
cmd = '/usr/bin/tar';
|
||||
args = ['-xzf', archivePath, '-C', destDir];
|
||||
}
|
||||
const child = spawn(cmd, args);
|
||||
let stderr = '';
|
||||
child.stderr.on('data', (c) => { stderr += c.toString(); });
|
||||
child.on('close', (code) => {
|
||||
if (code === 0) resolve();
|
||||
else reject(new Error(`${cmd} exited ${code}: ${stderr}`));
|
||||
});
|
||||
child.on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
function cloneGitRepo({ url, branch, destDir, logPath }) {
|
||||
return new Promise((resolve, reject) => {
|
||||
fs.mkdirSync(destDir, { recursive: true });
|
||||
const args = ['clone', '--depth', '1'];
|
||||
if (branch) args.push('--branch', branch);
|
||||
args.push(url, destDir);
|
||||
fs.appendFileSync(logPath, `$ git ${args.join(' ')}\n`);
|
||||
const child = spawn('/usr/bin/git', args);
|
||||
child.stdout.on('data', (c) => fs.appendFileSync(logPath, c));
|
||||
child.stderr.on('data', (c) => fs.appendFileSync(logPath, c));
|
||||
child.on('close', (code) => code === 0 ? resolve() : reject(new Error(`git clone failed (${code})`)));
|
||||
child.on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
function register(app, { requireLogin }) {
|
||||
// --- Pages ---
|
||||
app.get('/build', requireLogin, (req, res) => {
|
||||
res.sendFile(path.join(__dirname, '..', 'views', 'build.html'));
|
||||
});
|
||||
app.get('/builds', requireLogin, (req, res) => {
|
||||
res.sendFile(path.join(__dirname, '..', 'views', 'builds.html'));
|
||||
});
|
||||
|
||||
// --- Upload a source archive ---
|
||||
app.post('/api/build/upload', requireLogin, archiveUpload.single('source'), async (req, res) => {
|
||||
if (!req.file) return res.status(400).json({ error: 'No source file provided' });
|
||||
try {
|
||||
const jobId = uuidv4();
|
||||
const sourceDir = path.join(SOURCE_DIR, jobId);
|
||||
await extractArchive(req.file.path, sourceDir);
|
||||
fs.unlinkSync(req.file.path);
|
||||
|
||||
const scheme = (req.body.scheme || '').trim() || null;
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO build_jobs (id, source_kind, source_ref, scheme, status)
|
||||
VALUES (?, 'upload', ?, ?, 'pending')
|
||||
`).run(jobId, req.file.originalname, scheme);
|
||||
|
||||
buildWorker.kick();
|
||||
res.json({ success: true, job_id: jobId });
|
||||
} catch (err) {
|
||||
if (req.file && fs.existsSync(req.file.path)) fs.unlinkSync(req.file.path);
|
||||
res.status(500).json({ error: err.message });
|
||||
}
|
||||
});
|
||||
|
||||
// --- Clone a git repo ---
|
||||
app.post('/api/build/git', requireLogin, async (req, res) => {
|
||||
const { url, branch, scheme } = req.body || {};
|
||||
if (!url) return res.status(400).json({ error: 'url is required' });
|
||||
|
||||
const jobId = uuidv4();
|
||||
const logPath = path.join(LOGS_DIR, `${jobId}.log`);
|
||||
fs.writeFileSync(logPath, `Cloning ${url}${branch ? ` (branch ${branch})` : ''}…\n`);
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO build_jobs (id, source_kind, source_ref, scheme, status, log_path)
|
||||
VALUES (?, 'git', ?, ?, 'pending', ?)
|
||||
`).run(jobId, url, scheme || null, logPath);
|
||||
|
||||
try {
|
||||
const sourceDir = path.join(SOURCE_DIR, jobId);
|
||||
await cloneGitRepo({ url, branch, destDir: sourceDir, logPath });
|
||||
buildWorker.kick();
|
||||
res.json({ success: true, job_id: jobId });
|
||||
} catch (err) {
|
||||
db.prepare("UPDATE build_jobs SET status = 'failed', error = ?, finished_at = datetime('now') WHERE id = ?")
|
||||
.run(err.message, jobId);
|
||||
res.status(500).json({ error: err.message });
|
||||
}
|
||||
});
|
||||
|
||||
// --- List jobs ---
|
||||
app.get('/api/builds', requireLogin, (req, res) => {
|
||||
const rows = db.prepare(`
|
||||
SELECT id, bundle_id, source_kind, source_ref, scheme, status, started_at, finished_at, error, unraid_build_id, install_url
|
||||
FROM build_jobs
|
||||
ORDER BY COALESCE(started_at, created_at) DESC
|
||||
LIMIT 100
|
||||
`).all();
|
||||
res.json(rows);
|
||||
});
|
||||
|
||||
// --- Get a single job ---
|
||||
app.get('/api/builds/:id', requireLogin, (req, res) => {
|
||||
const row = db.prepare('SELECT * FROM build_jobs WHERE id = ?').get(req.params.id);
|
||||
if (!row) return res.status(404).json({ error: 'not found' });
|
||||
res.json(row);
|
||||
});
|
||||
|
||||
// --- Stream logs via SSE ---
|
||||
app.get('/api/builds/:id/logs', requireLogin, (req, res) => {
|
||||
const row = db.prepare('SELECT * FROM build_jobs WHERE id = ?').get(req.params.id);
|
||||
if (!row) return res.status(404).end();
|
||||
const logPath = row.log_path || path.join(LOGS_DIR, `${row.id}.log`);
|
||||
|
||||
res.set({
|
||||
'Content-Type': 'text/event-stream',
|
||||
'Cache-Control': 'no-cache',
|
||||
Connection: 'keep-alive',
|
||||
});
|
||||
res.flushHeaders?.();
|
||||
|
||||
let position = 0;
|
||||
const sendNew = () => {
|
||||
if (!fs.existsSync(logPath)) return;
|
||||
const stat = fs.statSync(logPath);
|
||||
if (stat.size <= position) return;
|
||||
const fd = fs.openSync(logPath, 'r');
|
||||
const buf = Buffer.alloc(stat.size - position);
|
||||
fs.readSync(fd, buf, 0, buf.length, position);
|
||||
fs.closeSync(fd);
|
||||
position = stat.size;
|
||||
// SSE: prefix every line with "data: "
|
||||
const lines = buf.toString('utf8').split('\n');
|
||||
for (const line of lines) {
|
||||
if (line.length) res.write(`data: ${line}\n\n`);
|
||||
}
|
||||
};
|
||||
|
||||
sendNew();
|
||||
const interval = setInterval(() => {
|
||||
sendNew();
|
||||
// Check if job finished — send one more time and close after a grace period.
|
||||
const current = db.prepare('SELECT status FROM build_jobs WHERE id = ?').get(req.params.id);
|
||||
if (current && (current.status === 'succeeded' || current.status === 'failed')) {
|
||||
sendNew();
|
||||
res.write(`event: done\ndata: ${current.status}\n\n`);
|
||||
clearInterval(interval);
|
||||
res.end();
|
||||
}
|
||||
}, 1000);
|
||||
|
||||
req.on('close', () => clearInterval(interval));
|
||||
});
|
||||
|
||||
// --- Rebuild a finished job (reuses the last known source if available) ---
|
||||
app.post('/api/builds/:id/rebuild', requireLogin, (req, res) => {
|
||||
res.status(501).json({ error: 'rebuild not implemented yet' });
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = { register };
|
||||
334
builder/src/build-worker.js
Normal file
334
builder/src/build-worker.js
Normal file
@@ -0,0 +1,334 @@
|
||||
// Build worker — consumes `build_jobs` rows, runs xcodebuild + fastlane + upload.
|
||||
// Single in-process loop; SQLite is the queue.
|
||||
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const os = require('os');
|
||||
const { execFile, spawn } = require('child_process');
|
||||
const { promisify } = require('util');
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
const { db, getSetting, DATA_DIR } = require('./db');
|
||||
const profileManager = require('./profile-manager');
|
||||
|
||||
const SOURCE_DIR = path.join(DATA_DIR, 'source');
|
||||
const BUILD_DIR = path.join(DATA_DIR, 'build');
|
||||
const LOGS_DIR = path.join(DATA_DIR, 'builds');
|
||||
|
||||
[SOURCE_DIR, BUILD_DIR, LOGS_DIR].forEach((d) => fs.mkdirSync(d, { recursive: true }));
|
||||
|
||||
const POLL_INTERVAL_MS = 2000;
|
||||
let running = false;
|
||||
|
||||
// --- Job state helpers ---
|
||||
|
||||
function markStatus(jobId, status, extra = {}) {
|
||||
const fields = [];
|
||||
const values = [];
|
||||
fields.push('status = ?'); values.push(status);
|
||||
for (const [k, v] of Object.entries(extra)) {
|
||||
fields.push(`${k} = ?`);
|
||||
values.push(v);
|
||||
}
|
||||
if (status === 'failed' || status === 'succeeded') {
|
||||
fields.push("finished_at = datetime('now')");
|
||||
}
|
||||
values.push(jobId);
|
||||
db.prepare(`UPDATE build_jobs SET ${fields.join(', ')} WHERE id = ?`).run(...values);
|
||||
}
|
||||
|
||||
function appendLog(logPath, line) {
|
||||
fs.appendFileSync(logPath, line.endsWith('\n') ? line : line + '\n');
|
||||
}
|
||||
|
||||
function section(logPath, title) {
|
||||
appendLog(logPath, `\n━━━━━━━━ ${title} ━━━━━━━━`);
|
||||
}
|
||||
|
||||
// --- Command runner that streams to the log file ---
|
||||
|
||||
function runCommand(cmd, args, { cwd, env, logPath }) {
|
||||
return new Promise((resolve, reject) => {
|
||||
appendLog(logPath, `$ ${cmd} ${args.join(' ')}`);
|
||||
const child = spawn(cmd, args, { cwd, env });
|
||||
let stdout = '', stderr = '';
|
||||
child.stdout.on('data', (c) => { const s = c.toString(); stdout += s; fs.appendFileSync(logPath, s); });
|
||||
child.stderr.on('data', (c) => { const s = c.toString(); stderr += s; fs.appendFileSync(logPath, s); });
|
||||
child.on('error', reject);
|
||||
child.on('close', (code) => {
|
||||
if (code === 0) resolve({ stdout, stderr });
|
||||
else reject(new Error(`${cmd} exited with code ${code}`));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// --- Project locator ---
|
||||
|
||||
function findProjectRoot(sourceDir) {
|
||||
// Walk up to 3 levels looking for .xcodeproj/.xcworkspace.
|
||||
const walk = (dir, depth) => {
|
||||
if (depth > 3) return null;
|
||||
const entries = fs.readdirSync(dir, { withFileTypes: true });
|
||||
const workspace = entries.find((e) => e.isDirectory() && e.name.endsWith('.xcworkspace') && !e.name.endsWith('.xcodeproj/project.xcworkspace'));
|
||||
if (workspace) return { dir, type: 'workspace', name: workspace.name };
|
||||
const project = entries.find((e) => e.isDirectory() && e.name.endsWith('.xcodeproj'));
|
||||
if (project) return { dir, type: 'project', name: project.name };
|
||||
for (const e of entries) {
|
||||
if (e.isDirectory() && !e.name.startsWith('.') && e.name !== 'node_modules') {
|
||||
const found = walk(path.join(dir, e.name), depth + 1);
|
||||
if (found) return found;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
};
|
||||
return walk(sourceDir, 0);
|
||||
}
|
||||
|
||||
async function listSchemes({ projectRoot, logPath }) {
|
||||
const args = projectRoot.type === 'workspace'
|
||||
? ['-list', '-json', '-workspace', projectRoot.name]
|
||||
: ['-list', '-json', '-project', projectRoot.name];
|
||||
const { stdout } = await runCommand('/usr/bin/xcodebuild', args, { cwd: projectRoot.dir, env: process.env, logPath });
|
||||
try {
|
||||
const parsed = JSON.parse(stdout);
|
||||
return parsed.workspace?.schemes || parsed.project?.schemes || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
async function getBuildSettings({ projectRoot, scheme, logPath }) {
|
||||
const args = ['-showBuildSettings', '-json', '-scheme', scheme];
|
||||
if (projectRoot.type === 'workspace') args.unshift('-workspace', projectRoot.name);
|
||||
else args.unshift('-project', projectRoot.name);
|
||||
const { stdout } = await runCommand('/usr/bin/xcodebuild', args, { cwd: projectRoot.dir, env: process.env, logPath });
|
||||
try {
|
||||
return JSON.parse(stdout);
|
||||
} catch {
|
||||
throw new Error('Could not parse xcodebuild -showBuildSettings JSON');
|
||||
}
|
||||
}
|
||||
|
||||
function extractBundleIds(buildSettings) {
|
||||
const bundleIds = new Set();
|
||||
let teamId = null;
|
||||
for (const target of buildSettings) {
|
||||
const s = target.buildSettings || {};
|
||||
if (s.PRODUCT_BUNDLE_IDENTIFIER) bundleIds.add(s.PRODUCT_BUNDLE_IDENTIFIER);
|
||||
if (s.DEVELOPMENT_TEAM && !teamId) teamId = s.DEVELOPMENT_TEAM;
|
||||
}
|
||||
return { bundleIds: Array.from(bundleIds), teamId };
|
||||
}
|
||||
|
||||
// --- ExportOptions.plist generation ---
|
||||
|
||||
function buildExportOptions({ teamId, profilesByBundleId }) {
|
||||
const entries = Object.entries(profilesByBundleId)
|
||||
.map(([bid, uuid]) => ` <key>${bid}</key><string>${uuid}</string>`)
|
||||
.join('\n');
|
||||
return `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>method</key><string>ad-hoc</string>
|
||||
<key>teamID</key><string>${teamId}</string>
|
||||
<key>signingStyle</key><string>manual</string>
|
||||
<key>stripSwiftSymbols</key><true/>
|
||||
<key>compileBitcode</key><false/>
|
||||
<key>provisioningProfiles</key>
|
||||
<dict>
|
||||
${entries}
|
||||
</dict>
|
||||
</dict>
|
||||
</plist>
|
||||
`;
|
||||
}
|
||||
|
||||
// --- Upload to unraid ---
|
||||
|
||||
async function uploadToUnraid({ ipaPath, notes, logPath }) {
|
||||
const unraidUrl = getSetting('unraid_url');
|
||||
const unraidToken = getSetting('unraid_token');
|
||||
if (!unraidUrl || !unraidToken) throw new Error('unraid URL/token not configured');
|
||||
|
||||
appendLog(logPath, `\nUploading IPA to ${unraidUrl}/api/upload`);
|
||||
|
||||
const buf = fs.readFileSync(ipaPath);
|
||||
const blob = new Blob([buf], { type: 'application/octet-stream' });
|
||||
const form = new FormData();
|
||||
form.append('ipa', blob, path.basename(ipaPath));
|
||||
if (notes) form.append('notes', notes);
|
||||
|
||||
const res = await fetch(`${unraidUrl}/api/upload`, {
|
||||
method: 'POST',
|
||||
headers: { 'X-Api-Token': unraidToken },
|
||||
body: form,
|
||||
});
|
||||
const body = await res.json().catch(() => ({}));
|
||||
if (!res.ok || !body.success) {
|
||||
throw new Error(`unraid upload failed (${res.status}): ${JSON.stringify(body)}`);
|
||||
}
|
||||
appendLog(logPath, `✓ Uploaded: ${JSON.stringify(body)}`);
|
||||
return body;
|
||||
}
|
||||
|
||||
// --- Main build function ---
|
||||
|
||||
async function runBuild(job) {
|
||||
const jobId = job.id;
|
||||
const logPath = path.join(LOGS_DIR, `${jobId}.log`);
|
||||
markStatus(jobId, 'preparing', { log_path: logPath, started_at: "datetime('now')" });
|
||||
// Reset started_at properly (datetime() doesn't work via binding above).
|
||||
db.prepare("UPDATE build_jobs SET started_at = datetime('now') WHERE id = ?").run(jobId);
|
||||
|
||||
fs.writeFileSync(logPath, `Build ${jobId} started at ${new Date().toISOString()}\n`);
|
||||
section(logPath, 'PREPARING');
|
||||
|
||||
const sourceDir = path.join(SOURCE_DIR, jobId);
|
||||
if (!fs.existsSync(sourceDir)) throw new Error(`source dir missing: ${sourceDir}`);
|
||||
|
||||
const projectRoot = findProjectRoot(sourceDir);
|
||||
if (!projectRoot) throw new Error('No .xcodeproj or .xcworkspace found in source');
|
||||
appendLog(logPath, `Found ${projectRoot.type}: ${projectRoot.name} in ${projectRoot.dir}`);
|
||||
|
||||
// Pick the scheme
|
||||
const schemes = await listSchemes({ projectRoot, logPath });
|
||||
if (!schemes.length) throw new Error('No schemes found in project');
|
||||
const scheme = job.scheme && schemes.includes(job.scheme)
|
||||
? job.scheme
|
||||
: schemes[0];
|
||||
appendLog(logPath, `Using scheme: ${scheme} (available: ${schemes.join(', ')})`);
|
||||
|
||||
// Read build settings for bundle IDs + team id
|
||||
const buildSettings = await getBuildSettings({ projectRoot, scheme, logPath });
|
||||
const { bundleIds, teamId } = extractBundleIds(buildSettings);
|
||||
if (!bundleIds.length) throw new Error('Could not determine bundle identifiers');
|
||||
if (!teamId) throw new Error('Could not determine development team ID');
|
||||
appendLog(logPath, `Team: ${teamId}`);
|
||||
appendLog(logPath, `Bundle IDs: ${bundleIds.join(', ')}`);
|
||||
|
||||
// Persist the primary bundle id on the job row
|
||||
db.prepare('UPDATE build_jobs SET bundle_id = ?, scheme = ? WHERE id = ?')
|
||||
.run(bundleIds[0], scheme, jobId);
|
||||
|
||||
// --- Signing phase ---
|
||||
section(logPath, 'SIGNING');
|
||||
markStatus(jobId, 'signing');
|
||||
|
||||
const profilesByBundleId = {};
|
||||
for (const bid of bundleIds) {
|
||||
appendLog(logPath, `Ensuring profile for ${bid}…`);
|
||||
const info = await profileManager.getProfile(bid);
|
||||
profilesByBundleId[bid] = info.profile_uuid;
|
||||
appendLog(logPath, ` → ${info.profile_uuid} (${info.fromCache ? 'cache' : 'fresh'}, ${info.device_count} devices, expires ${info.expires_at})`);
|
||||
}
|
||||
|
||||
// --- Archiving phase ---
|
||||
section(logPath, 'ARCHIVING');
|
||||
markStatus(jobId, 'archiving');
|
||||
|
||||
const archivePath = path.join(BUILD_DIR, `${jobId}.xcarchive`);
|
||||
const archiveArgs = [
|
||||
projectRoot.type === 'workspace' ? '-workspace' : '-project', projectRoot.name,
|
||||
'-scheme', scheme,
|
||||
'-configuration', 'Release',
|
||||
'-destination', 'generic/platform=iOS',
|
||||
'-archivePath', archivePath,
|
||||
'-allowProvisioningUpdates',
|
||||
'CODE_SIGN_STYLE=Manual',
|
||||
`DEVELOPMENT_TEAM=${teamId}`,
|
||||
'archive',
|
||||
];
|
||||
// We can't specify per-target PROVISIONING_PROFILE_SPECIFIER globally, so we rely on
|
||||
// xcodebuild finding the installed profiles in ~/Library/MobileDevice/Provisioning Profiles/
|
||||
// by matching bundle id + team id.
|
||||
await runCommand('/usr/bin/xcodebuild', archiveArgs, { cwd: projectRoot.dir, env: process.env, logPath });
|
||||
|
||||
// --- Exporting phase ---
|
||||
section(logPath, 'EXPORTING');
|
||||
markStatus(jobId, 'exporting');
|
||||
|
||||
const exportPath = path.join(BUILD_DIR, jobId);
|
||||
fs.mkdirSync(exportPath, { recursive: true });
|
||||
|
||||
const exportOptionsPath = path.join(exportPath, 'ExportOptions.plist');
|
||||
fs.writeFileSync(exportOptionsPath, buildExportOptions({ teamId, profilesByBundleId }));
|
||||
appendLog(logPath, `Wrote ExportOptions.plist:\n${fs.readFileSync(exportOptionsPath, 'utf8')}`);
|
||||
|
||||
const exportArgs = [
|
||||
'-exportArchive',
|
||||
'-archivePath', archivePath,
|
||||
'-exportPath', exportPath,
|
||||
'-exportOptionsPlist', exportOptionsPath,
|
||||
'-allowProvisioningUpdates',
|
||||
];
|
||||
await runCommand('/usr/bin/xcodebuild', exportArgs, { cwd: projectRoot.dir, env: process.env, logPath });
|
||||
|
||||
// Find the produced IPA
|
||||
const ipaFile = fs.readdirSync(exportPath).find((f) => f.endsWith('.ipa'));
|
||||
if (!ipaFile) throw new Error('Export succeeded but no .ipa produced');
|
||||
const ipaPath = path.join(exportPath, ipaFile);
|
||||
appendLog(logPath, `IPA produced: ${ipaPath} (${(fs.statSync(ipaPath).size / (1024*1024)).toFixed(1)} MB)`);
|
||||
|
||||
// --- Uploading phase ---
|
||||
section(logPath, 'UPLOADING');
|
||||
markStatus(jobId, 'uploading', { ipa_path: ipaPath });
|
||||
|
||||
const uploadResult = await uploadToUnraid({
|
||||
ipaPath,
|
||||
notes: `Built by ${os.hostname()} job ${jobId}`,
|
||||
logPath,
|
||||
});
|
||||
|
||||
markStatus(jobId, 'succeeded', {
|
||||
unraid_build_id: uploadResult.build?.id || null,
|
||||
install_url: uploadResult.build?.install_url || null,
|
||||
});
|
||||
|
||||
// --- Cleanup: keep log + IPA, remove source + archive ---
|
||||
try {
|
||||
fs.rmSync(sourceDir, { recursive: true, force: true });
|
||||
fs.rmSync(archivePath, { recursive: true, force: true });
|
||||
} catch (e) {
|
||||
appendLog(logPath, `Cleanup warning: ${e.message}`);
|
||||
}
|
||||
|
||||
section(logPath, `SUCCEEDED at ${new Date().toISOString()}`);
|
||||
}
|
||||
|
||||
async function processJob(job) {
|
||||
const logPath = path.join(LOGS_DIR, `${job.id}.log`);
|
||||
try {
|
||||
await runBuild(job);
|
||||
} catch (err) {
|
||||
console.error(`[build-worker] job ${job.id} failed:`, err);
|
||||
try { appendLog(logPath, `\n✗ FAILED: ${err.message}\n${err.stack || ''}`); } catch {}
|
||||
markStatus(job.id, 'failed', { error: err.message });
|
||||
}
|
||||
}
|
||||
|
||||
async function loop() {
|
||||
if (running) return;
|
||||
running = true;
|
||||
try {
|
||||
while (true) {
|
||||
const job = db.prepare("SELECT * FROM build_jobs WHERE status = 'pending' ORDER BY created_at ASC LIMIT 1").get();
|
||||
if (!job) break;
|
||||
await processJob(job);
|
||||
}
|
||||
} finally {
|
||||
running = false;
|
||||
}
|
||||
}
|
||||
|
||||
function kick() {
|
||||
// Non-blocking: fire and forget.
|
||||
loop().catch((err) => console.error('[build-worker] loop error:', err));
|
||||
}
|
||||
|
||||
function start() {
|
||||
setInterval(kick, POLL_INTERVAL_MS);
|
||||
kick();
|
||||
}
|
||||
|
||||
module.exports = { start, kick, runBuild, processJob };
|
||||
@@ -21,7 +21,7 @@ const path = require('path');
|
||||
const fs = require('fs');
|
||||
|
||||
const { db, getSetting, setSetting, DATA_DIR } = require('./db');
|
||||
const { requireLogin, validatePassword } = require('./auth');
|
||||
const { requireLogin, requireBuilderSecret, validatePassword } = require('./auth');
|
||||
|
||||
const app = express();
|
||||
const PORT = process.env.PORT || 3090;
|
||||
@@ -71,7 +71,7 @@ app.get('/logout', (req, res) => {
|
||||
// --- Pages ---
|
||||
|
||||
app.get('/', requireLogin, (req, res) => {
|
||||
res.sendFile(path.join(__dirname, '..', 'views', 'index.html'));
|
||||
res.sendFile(path.join(__dirname, '..', 'views', 'builds.html'));
|
||||
});
|
||||
|
||||
app.get('/settings', requireLogin, (req, res) => {
|
||||
@@ -137,6 +137,43 @@ app.delete('/api/devices/:udid', requireLogin, (req, res) => {
|
||||
res.json({ success: true });
|
||||
});
|
||||
|
||||
// --- Enrollment bridge (called by unraid's /enroll/callback over the LAN) ---
|
||||
|
||||
app.post('/api/devices/from-enrollment', requireBuilderSecret, async (req, res) => {
|
||||
const { udid, name, model, platform = 'IOS' } = req.body || {};
|
||||
if (!udid || typeof udid !== 'string') {
|
||||
return res.status(400).json({ error: 'UDID is required' });
|
||||
}
|
||||
|
||||
db.prepare(`
|
||||
INSERT INTO devices (udid, name, model, platform)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(udid) DO UPDATE SET
|
||||
name = COALESCE(NULLIF(excluded.name, ''), devices.name),
|
||||
model = COALESCE(NULLIF(excluded.model, ''), devices.model),
|
||||
platform = excluded.platform
|
||||
`).run(udid, name || null, model || null, platform);
|
||||
|
||||
let synced = false;
|
||||
try {
|
||||
const asc = require('./asc-api');
|
||||
const appleDevice = await asc.registerDevice({ udid, name, platform });
|
||||
const appleDeviceId = appleDevice?.id || null;
|
||||
db.prepare(`
|
||||
UPDATE devices
|
||||
SET apple_device_id = ?, synced_at = datetime('now')
|
||||
WHERE udid = ?
|
||||
`).run(appleDeviceId, udid);
|
||||
synced = true;
|
||||
invalidateProfilesForDeviceChange();
|
||||
} catch (err) {
|
||||
console.warn('[enrollment] ASC sync failed:', err.message);
|
||||
return res.json({ success: true, synced: false, warning: err.message });
|
||||
}
|
||||
|
||||
res.json({ success: true, synced });
|
||||
});
|
||||
|
||||
// --- Settings API ---
|
||||
|
||||
const SETTINGS_KEYS = [
|
||||
@@ -215,6 +252,10 @@ app.post('/api/settings/test-unraid', requireLogin, async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
// --- Build pipeline ---
|
||||
|
||||
require('./build-routes').register(app, { requireLogin });
|
||||
|
||||
// --- Profile API ---
|
||||
|
||||
app.get('/api/profile/:bundleId', requireLogin, async (req, res) => {
|
||||
@@ -248,4 +289,7 @@ app.get('/api/health', (req, res) => {
|
||||
app.listen(PORT, '0.0.0.0', () => {
|
||||
console.log(`iOS App Store Builder running on port ${PORT}`);
|
||||
console.log(`Data dir: ${DATA_DIR}`);
|
||||
// Start the build worker loop.
|
||||
require('./build-worker').start();
|
||||
console.log('Build worker started');
|
||||
});
|
||||
|
||||
67
builder/views/build.html
Normal file
67
builder/views/build.html
Normal file
@@ -0,0 +1,67 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>New Build - Builder</title>
|
||||
<link rel="stylesheet" href="/css/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<div class="header-left"><h1>🔨 Builder</h1></div>
|
||||
<nav>
|
||||
<a href="/">Builds</a>
|
||||
<a href="/build" class="active">New Build</a>
|
||||
<a href="/devices">Devices</a>
|
||||
<a href="/settings">Settings</a>
|
||||
<a href="/logout" class="logout">Logout</a>
|
||||
</nav>
|
||||
</header>
|
||||
|
||||
<main>
|
||||
<h1 class="page-title">New Build</h1>
|
||||
|
||||
<div class="section">
|
||||
<h2>From source archive</h2>
|
||||
<div class="card">
|
||||
<form id="upload-form" enctype="multipart/form-data">
|
||||
<label>Archive (.zip or .tar.gz)</label>
|
||||
<input type="file" name="source" id="source-input" accept=".zip,.tar.gz,.tgz" required>
|
||||
<label>Scheme (optional)</label>
|
||||
<input type="text" name="scheme" placeholder="leave blank to use the first scheme">
|
||||
<div class="btn-row">
|
||||
<button type="submit">Queue Build</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>From git URL</h2>
|
||||
<div class="card">
|
||||
<form id="git-form">
|
||||
<label>Repository URL</label>
|
||||
<input type="text" name="url" placeholder="git@gitea.treytartt.com:user/repo.git or https://…">
|
||||
<div class="field-group">
|
||||
<div>
|
||||
<label>Branch (optional)</label>
|
||||
<input type="text" name="branch" placeholder="main">
|
||||
</div>
|
||||
<div>
|
||||
<label>Scheme (optional)</label>
|
||||
<input type="text" name="scheme" placeholder="first scheme">
|
||||
</div>
|
||||
</div>
|
||||
<div class="btn-row">
|
||||
<button type="submit">Queue Build</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="toast" class="toast"></div>
|
||||
</main>
|
||||
|
||||
<script src="/js/build.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
40
builder/views/builds.html
Normal file
40
builder/views/builds.html
Normal file
@@ -0,0 +1,40 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Builds - Builder</title>
|
||||
<link rel="stylesheet" href="/css/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<div class="header-left"><h1>🔨 Builder</h1></div>
|
||||
<nav>
|
||||
<a href="/" class="active">Builds</a>
|
||||
<a href="/build">New Build</a>
|
||||
<a href="/devices">Devices</a>
|
||||
<a href="/settings">Settings</a>
|
||||
<a href="/logout" class="logout">Logout</a>
|
||||
</nav>
|
||||
</header>
|
||||
|
||||
<main>
|
||||
<div style="display:flex;align-items:center;justify-content:space-between;margin-bottom:16px">
|
||||
<h1 class="page-title" style="margin-bottom:0">Builds</h1>
|
||||
<a href="/build" class="btn-sm" style="background:var(--accent);color:white;padding:8px 16px;border-radius:20px;text-decoration:none;font-weight:600;font-size:14px">+ New Build</a>
|
||||
</div>
|
||||
|
||||
<div id="jobs-container"><div class="card"><p style="color:var(--text-muted)">Loading…</p></div></div>
|
||||
|
||||
<div id="detail" class="section" style="display:none">
|
||||
<h2 id="detail-title">Job details</h2>
|
||||
<div class="card">
|
||||
<div id="detail-meta" class="mono" style="font-size:12px;color:var(--text-muted);margin-bottom:12px"></div>
|
||||
<pre id="log-viewer" class="log-viewer"></pre>
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
|
||||
<script src="/js/builds.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -11,5 +11,7 @@ services:
|
||||
- SESSION_SECRET=${SESSION_SECRET}
|
||||
- BASE_URL=https://appstore.treytartt.com
|
||||
- DATA_DIR=/data
|
||||
- BUILDER_URL=${BUILDER_URL}
|
||||
- BUILDER_SHARED_SECRET=${BUILDER_SHARED_SECRET}
|
||||
volumes:
|
||||
- /mnt/user/downloads/ios-appstore:/data
|
||||
|
||||
87
src/mobileconfig.js
Normal file
87
src/mobileconfig.js
Normal file
@@ -0,0 +1,87 @@
|
||||
// Generates the Apple .mobileconfig Profile Service payload used by /enroll.
|
||||
// When a user installs this on iOS, the device POSTs back a signed plist
|
||||
// containing the UDID and other device attributes to the configured URL.
|
||||
// The plist we return is unsigned — iOS will show a "Not Signed" warning but
|
||||
// still allow installation, which is acceptable for an internal store.
|
||||
|
||||
const crypto = require('crypto');
|
||||
|
||||
function buildMobileConfig({ baseUrl, displayName = 'App Store Enrollment', organization = 'App Store' }) {
|
||||
// Create two random UUIDs for the PayloadUUID and PayloadIdentifier fields.
|
||||
// iOS uses these to uniquely identify the profile on the device.
|
||||
const profileUuid = crypto.randomUUID().toUpperCase();
|
||||
const payloadUuid = crypto.randomUUID().toUpperCase();
|
||||
|
||||
const callbackUrl = `${baseUrl}/enroll/callback`;
|
||||
|
||||
return `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>PayloadContent</key>
|
||||
<dict>
|
||||
<key>URL</key>
|
||||
<string>${callbackUrl}</string>
|
||||
<key>DeviceAttributes</key>
|
||||
<array>
|
||||
<string>UDID</string>
|
||||
<string>PRODUCT</string>
|
||||
<string>VERSION</string>
|
||||
<string>DEVICE_NAME</string>
|
||||
<string>SERIAL</string>
|
||||
</array>
|
||||
</dict>
|
||||
<key>PayloadOrganization</key>
|
||||
<string>${escapeXml(organization)}</string>
|
||||
<key>PayloadDisplayName</key>
|
||||
<string>${escapeXml(displayName)}</string>
|
||||
<key>PayloadVersion</key>
|
||||
<integer>1</integer>
|
||||
<key>PayloadUUID</key>
|
||||
<string>${profileUuid}</string>
|
||||
<key>PayloadIdentifier</key>
|
||||
<string>com.88oak.appstore.enroll.${payloadUuid}</string>
|
||||
<key>PayloadDescription</key>
|
||||
<string>Register this device with the internal app store.</string>
|
||||
<key>PayloadType</key>
|
||||
<string>Profile Service</string>
|
||||
</dict>
|
||||
</plist>
|
||||
`;
|
||||
}
|
||||
|
||||
function escapeXml(s) {
|
||||
return String(s).replace(/[<>&"']/g, (c) => ({
|
||||
'<': '<', '>': '>', '&': '&', '"': '"', "'": '''
|
||||
}[c]));
|
||||
}
|
||||
|
||||
// Parse the CMS-wrapped plist iOS posts back. We don't validate the signature —
|
||||
// we just extract the inner XML plist and pull the UDID / PRODUCT / etc out of it.
|
||||
function parseEnrollmentCallback(rawBody) {
|
||||
// iOS sends application/pkcs7-signature with a CMS SignedData envelope.
|
||||
// The inner plist is ASCII; locate it by its XML markers.
|
||||
const start = rawBody.indexOf('<?xml');
|
||||
const end = rawBody.indexOf('</plist>');
|
||||
if (start === -1 || end === -1) {
|
||||
throw new Error('Could not find inner plist in enrollment payload');
|
||||
}
|
||||
const xml = rawBody.slice(start, end + '</plist>'.length).toString('utf8');
|
||||
|
||||
const pick = (key) => {
|
||||
const re = new RegExp(`<key>${key}</key>\\s*<(?:string|data)>([^<]+)</(?:string|data)>`);
|
||||
const m = xml.match(re);
|
||||
return m ? m[1].trim() : null;
|
||||
};
|
||||
|
||||
return {
|
||||
udid: pick('UDID'),
|
||||
product: pick('PRODUCT'),
|
||||
version: pick('VERSION'),
|
||||
deviceName: pick('DEVICE_NAME'),
|
||||
serial: pick('SERIAL'),
|
||||
rawXml: xml,
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = { buildMobileConfig, parseEnrollmentCallback };
|
||||
@@ -248,6 +248,75 @@ app.delete('/api/devices/:udid', requireAuth, (req, res) => {
|
||||
res.json({ success: true });
|
||||
});
|
||||
|
||||
// --- Enrollment (public — iOS hits these during profile install) ---
|
||||
|
||||
const { buildMobileConfig, parseEnrollmentCallback } = require('./mobileconfig');
|
||||
|
||||
app.get('/enroll', (req, res) => {
|
||||
res.sendFile(path.join(__dirname, '..', 'views', 'enroll.html'));
|
||||
});
|
||||
|
||||
app.get('/enroll/success', (req, res) => {
|
||||
res.sendFile(path.join(__dirname, '..', 'views', 'enroll-success.html'));
|
||||
});
|
||||
|
||||
app.get('/enroll/profile.mobileconfig', (req, res) => {
|
||||
const xml = buildMobileConfig({ baseUrl: BASE_URL });
|
||||
res.set('Content-Type', 'application/x-apple-aspen-config');
|
||||
res.set('Content-Disposition', 'attachment; filename="enroll.mobileconfig"');
|
||||
res.send(xml);
|
||||
});
|
||||
|
||||
// iOS posts a CMS-wrapped plist here after the user installs the profile.
|
||||
// We accept any Content-Type and parse the raw body for the inner plist.
|
||||
app.post('/enroll/callback',
|
||||
express.raw({ type: '*/*', limit: '256kb' }),
|
||||
async (req, res) => {
|
||||
try {
|
||||
const parsed = parseEnrollmentCallback(req.body);
|
||||
if (!parsed.udid) {
|
||||
console.warn('[enroll] callback without UDID');
|
||||
return res.status(400).send('Missing UDID');
|
||||
}
|
||||
|
||||
const builderUrl = process.env.BUILDER_URL;
|
||||
const sharedSecret = process.env.BUILDER_SHARED_SECRET;
|
||||
if (!builderUrl || !sharedSecret) {
|
||||
console.error('[enroll] BUILDER_URL / BUILDER_SHARED_SECRET not configured');
|
||||
// Still redirect the user so they don't see a broken state; log loudly.
|
||||
} else {
|
||||
try {
|
||||
const r = await fetch(`${builderUrl}/api/devices/from-enrollment`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Authorization: `Bearer ${sharedSecret}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
udid: parsed.udid,
|
||||
name: parsed.deviceName || parsed.product || null,
|
||||
model: parsed.product || null,
|
||||
}),
|
||||
});
|
||||
if (!r.ok) {
|
||||
console.error('[enroll] builder returned', r.status, await r.text());
|
||||
} else {
|
||||
console.log('[enroll] forwarded UDID to builder:', parsed.udid);
|
||||
}
|
||||
} catch (err) {
|
||||
console.error('[enroll] builder bridge failed:', err.message);
|
||||
}
|
||||
}
|
||||
|
||||
// iOS expects a 200 or a redirect to show the success page in Safari.
|
||||
res.redirect(303, '/enroll/success');
|
||||
} catch (err) {
|
||||
console.error('[enroll] callback parse failed:', err.message);
|
||||
res.status(400).send(`Invalid enrollment payload: ${err.message}`);
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// --- Health ---
|
||||
app.get('/api/health', (req, res) => {
|
||||
res.json({ status: 'ok', version: '1.0.0' });
|
||||
|
||||
20
views/enroll-success.html
Normal file
20
views/enroll-success.html
Normal file
@@ -0,0 +1,20 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Enrolled - App Store</title>
|
||||
<link rel="stylesheet" href="/css/style.css">
|
||||
</head>
|
||||
<body class="login-page">
|
||||
<div class="login-card">
|
||||
<div class="login-icon">✅</div>
|
||||
<h1>You're Enrolled</h1>
|
||||
<p class="subtitle">Your device has been registered</p>
|
||||
<p style="color: var(--text-muted); font-size: 13px; line-height: 1.5; margin-bottom: 20px">
|
||||
Your UDID has been added to the app store. New builds will include your device automatically. You can now return to <a href="/" style="color: var(--accent)">the app store</a> to install apps.
|
||||
</p>
|
||||
<a href="/" class="btn" style="display:block;text-align:center">Browse Apps</a>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
23
views/enroll.html
Normal file
23
views/enroll.html
Normal file
@@ -0,0 +1,23 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Enroll - App Store</title>
|
||||
<link rel="stylesheet" href="/css/style.css">
|
||||
</head>
|
||||
<body class="login-page">
|
||||
<div class="login-card">
|
||||
<div class="login-icon">📱</div>
|
||||
<h1>Enroll Device</h1>
|
||||
<p class="subtitle">Register your iPhone/iPad with the app store</p>
|
||||
<p style="color: var(--text-muted); font-size: 13px; line-height: 1.5; margin-bottom: 20px">
|
||||
Tap the button below on your iOS device. iOS will ask you to install a configuration profile — accept it. Your device UDID will be registered automatically.
|
||||
</p>
|
||||
<a href="/enroll/profile.mobileconfig" class="btn" style="display:block;text-align:center">Install Profile</a>
|
||||
<p style="color: var(--text-muted); font-size: 11px; margin-top: 16px">
|
||||
You'll see a "Not Signed" warning — that's expected. Tap "Install" to continue.
|
||||
</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
Reference in New Issue
Block a user