Files
honeyDueAPI/docs/deployment/appendices/c-file-locations.md
T
Trey t 6f303dbbaa
Backend CI / Test (push) Has been cancelled
Backend CI / Contract Tests (push) Has been cancelled
Backend CI / Build (push) Has been cancelled
Backend CI / Lint (push) Has been cancelled
Backend CI / Secret Scanning (push) Has been cancelled
Migrate prod deploy from Swarm to K3s; add full deployment book
Infrastructure:
- Stack now runs on K3s v1.34.6 HA (3 Hetzner CX33 nodes as managers)
- Traefik DaemonSet + hostNetwork replaces Caddy + ingress mesh
- All manifests in deploy-k3s/manifests/; Swarm config (deploy/) kept
  temporarily for reference

Bug fixes surfaced during migration:
- Dockerfile: golang:1.24-alpine -> 1.25-alpine (go.mod requires 1.25)
- cache_service.go: remove sync.Once reassignment from inside Do()
  callback (was causing 'unlock of unlocked mutex' fatal after
  Redis Ping failure)
- router.go: relax CSP from 'default-src none' to 'default-src self'
  + allowlist fonts.googleapis.com so the marketing landing page CSS
  actually loads in browsers
- deploy/scripts/deploy_prod.sh: use docker buildx with
  --platform linux/amd64 so arm64 (Apple Silicon) dev machines produce
  images runnable on x86_64 Hetzner nodes; fix array expansion under
  set -u
- deploy/swarm-stack.prod.yml: fix secret source references to use
  top-level aliases (the '\${X_SECRET}' form never actually resolved);
  dozzle ports: long-form host_ip is rejected by Swarm, switched to
  short-form (bound to 0.0.0.0 with UFW-based loopback restriction);
  worker replicas 2 -> 1 (Asynq scheduler singleton)
- deploy-k3s/manifests/admin/deployment.yaml: probe path '/admin/' -> '/'
  (Next.js serves at root; /admin/ returned 404 and killed pods);
  startupProbe failureThreshold 12 -> 24
- deploy-k3s/manifests/pod-disruption-budgets.yaml: worker minAvailable
  1 -> 0 (singleton)
- deploy-k3s/manifests/api/deployment.yaml: startupProbe failureThreshold
  12 -> 48 (MigrateWithLock serializes across 3 replicas on first-boot;
  real startup takes up to 240s)
- .gitignore: tighten 'api' -> '/api' (was matching deploy-k3s/manifests/api/
  and admin/src/app/api/*, hiding legitimate files)

New files:
- deploy-k3s/manifests/traefik-helmchartconfig.yaml: DaemonSet +
  hostNetwork override for k3s-bundled Traefik
- deploy-k3s/manifests/ingress/ingress-simple.yaml: plain Ingress
  without TLS (CF Flexible SSL) and without middleware
- deploy-k3s/MIGRATION_NOTES.md: operator-facing migration log

Documentation:
- docs/deployment/ — full deployment book, 26 files, ~42k words:
  - Part I Overview, infrastructure, orchestrator choice (Ch 0-2)
  - Part II Networking, firewall, Cloudflare (Ch 3-4, 13)
  - Part III Security, Traefik ingress (Ch 5-6)
  - Part IV Services, DB, storage, secrets, registry (Ch 7-11)
  - Part V Data flow, deploy process, observability, failures, runbook
    (Ch 12, 14-17)
  - Part VI Cost, Swarm postmortem, roadmap (Ch 18-20)
  - Appendices: glossary, kubectl cheat sheet, file locations,
    consolidated citations
- README.md: Production Deployment section replaced with pointer to
  the book; Go version bumped to 1.25

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 07:20:54 -05:00

9.5 KiB

Appendix C — File Locations

Complete map of where every significant file lives — on the operator workstation, in the git repo, and on the Hetzner nodes.

Operator workstation

Kubernetes

Path Purpose
~/.kube/honeydue-k3s.yaml kubeconfig for the k3s cluster. Contains an admin bearer token. Mode 0600.
~/.kube/config Default kubeconfig (points elsewhere, not our cluster).

Set KUBECONFIG=~/.kube/honeydue-k3s.yaml before any kubectl command.

SSH

Path Purpose
~/.ssh/hetzner Private key for node SSH (ed25519). Mode 0600.
~/.ssh/hetzner.pub Public key corresponding to above.
~/.ssh/config Host aliases for hetzner1/hetzner2/hetzner3 → node IPs.

Public key content:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBU9xTTBD78tYUqHijgyU9PDqtmS4NuM/6uy8XgDzva+ hetzner2@myhoneydue.com

Docker

Path Purpose
~/.docker/config.json Docker CLI config. After docker login to Gitea, contains creds. Log out after each deploy to not leave PATs on disk.
~/Library/Containers/com.docker.docker/ Docker Desktop state (macOS).

Git repo (/Users/treyt/Desktop/code/honeyDue/honeyDueAPI-go/)

Top-level

Path Purpose
CLAUDE.md Project-wide instructions for Claude assistant. Never commit secrets here.
Dockerfile Multi-stage Docker build: api, worker, admin targets.
go.mod, go.sum Go module definition.
package.json (admin-ui/) Next.js dependencies.

Application code

Path Purpose
cmd/api/main.go API server entry point.
cmd/worker/main.go Background worker entry point.
cmd/admin/main.go (may or may not exist for Go admin variant)
internal/config/ Viper configuration loading.
internal/database/ Postgres connection, migrations.
internal/handlers/ HTTP handlers (one file per domain).
internal/services/ Business logic. cache_service.go is where the sync.Once bug was (Chapter 19).
internal/repositories/ GORM repositories.
internal/router/router.go Echo routes, including static file serving. CSP is set here.
internal/middleware/ Echo middleware (auth, logging, etc.).
internal/task/ Task predicates/scopes/categorization. See docs/TASK_LOGIC_ARCHITECTURE.md.

Deploy config (Swarm era — still exists, unused)

Path Purpose
deploy/ Legacy Swarm deploy root.
deploy/prod.env Non-secret config (ConfigMap source). Gitignored.
deploy/registry.env Gitea PAT + registry URL. Gitignored.
deploy/cluster.env Swarm cluster settings. Partly used for k3s too (manager host). Gitignored.
deploy/secrets/postgres_password.txt Neon password. Gitignored.
deploy/secrets/secret_key.txt App signing key (≥32 chars). Gitignored.
deploy/secrets/email_host_password.txt Fastmail password. Gitignored.
deploy/secrets/fcm_server_key.txt FCM key (placeholder, push off). Gitignored.
deploy/secrets/apns_auth_key.p8 APNs key (placeholder, push off). Gitignored.
deploy/swarm-stack.prod.yml Swarm stack definition. Unused after migration.
deploy/Caddyfile Caddy config. Unused after migration.
deploy/scripts/deploy_prod.sh Swarm deploy script. Unused.
deploy/DEPLOYING.md, deploy/README.md, deploy/shit_deploy_cant_do.md Swarm-era docs. Historical reference.

Deploy config (k3s)

Path Purpose
deploy-k3s/README.md k3s deployment README (scaffold version).
deploy-k3s/MIGRATION_NOTES.md Notes from Swarm → k3s migration.
deploy-k3s/SECURITY.md Security posture doc (scaffold).
deploy-k3s/config.yaml.example Template for a unified config.yaml (unused — we kept Swarm's file layout).
deploy-k3s/manifests/namespace.yaml Creates honeydue namespace.
deploy-k3s/manifests/rbac.yaml ServiceAccounts + automountServiceAccountToken: false.
deploy-k3s/manifests/pod-disruption-budgets.yaml PDBs for api (2/3) and worker (0/1).
deploy-k3s/manifests/network-policies.yaml Default-deny + allows. NOT currently applied.
deploy-k3s/manifests/api/deployment.yaml api Deployment.
deploy-k3s/manifests/api/service.yaml api ClusterIP Service.
deploy-k3s/manifests/api/hpa.yaml api HorizontalPodAutoscaler. NOT currently applied.
deploy-k3s/manifests/admin/deployment.yaml admin Deployment.
deploy-k3s/manifests/admin/service.yaml admin Service.
deploy-k3s/manifests/worker/deployment.yaml worker Deployment.
deploy-k3s/manifests/redis/deployment.yaml Redis Deployment.
deploy-k3s/manifests/redis/service.yaml Redis Service.
deploy-k3s/manifests/redis/pvc.yaml Redis PersistentVolumeClaim.
deploy-k3s/manifests/ingress/ingress.yaml Full Ingress with TLS + middleware (scaffold; needs CF origin cert).
deploy-k3s/manifests/ingress/ingress-simple.yaml Simple Ingress without TLS (what we actually apply).
deploy-k3s/manifests/ingress/middleware.yaml Traefik middleware CRDs. Not currently applied.
deploy-k3s/manifests/traefik-helmchartconfig.yaml Our DaemonSet + hostNetwork override for Traefik.
deploy-k3s/manifests/secrets.yaml.example Template (never deployed).
deploy-k3s/scripts/01-provision-cluster.sh hetzner-k3s provisioning (we didn't use it; existing nodes).
deploy-k3s/scripts/02-setup-secrets.sh Creates Secrets + ConfigMap (scaffold version; we ran commands manually).
deploy-k3s/scripts/03-deploy.sh Applies manifests (unused; we ran kubectl manually).
deploy-k3s/scripts/04-verify.sh Post-deploy verification.
deploy-k3s/scripts/rollback.sh Rollback helper.

Documentation

Path Purpose
docs/deployment/ This book.
docs/TASK_LOGIC_ARCHITECTURE.md Task logic internals.
docs/PUSH_NOTIFICATIONS.md Push notifications setup (for future).
docs/SUBSCRIPTION_WEBHOOKS.md Apple/Google subscription webhooks.
docs/Dokku_notes Pre-Swarm era deployment notes. Historical.
docs/server_2026_2_24.md Earlier architecture doc (predates k3s migration).

On the Hetzner nodes

System

Path Purpose
/etc/ssh/sshd_config SSH config — PermitRootLogin no, PasswordAuthentication no, AllowUsers deploy.
/etc/sudoers.d/deploy deploy ALL=(ALL) NOPASSWD: ALL.
/etc/ufw/ UFW configuration. See Chapter 4 for rule inventory.
/etc/sysctl.d/99-unprivileged-ports.conf net.ipv4.ip_unprivileged_port_start=0 for Traefik.
/home/deploy/.ssh/authorized_keys Our hetzner.pub.

K3s

Path Purpose
/etc/rancher/k3s/k3s.yaml Kubeconfig (localhost-scoped; we copied to workstation).
/etc/systemd/system/k3s.service systemd service file.
/etc/systemd/system/k3s.service.env K3s install args (INSTALL_K3S_EXEC).
/var/lib/rancher/k3s/ K3s state root (etcd, containerd, PVC storage).
/var/lib/rancher/k3s/server/node-token Token for joining additional nodes.
/var/lib/rancher/k3s/storage/ local-path PVC storage. Redis data lives here.
/var/lib/rancher/k3s/agent/containerd/ containerd state.
/var/log/containers/ Container log files.

Commands installed

Path Purpose
/usr/local/bin/k3s The k3s binary.
/usr/local/bin/kubectl Symlink to k3s (CLI for this cluster).
/usr/local/bin/crictl containerd CLI.
/usr/local/bin/k3s-killall.sh Emergency kill-all-k3s script.
/usr/local/bin/k3s-uninstall.sh Clean uninstall script.

Docker (legacy; disabled)

Path Purpose
/etc/systemd/system/docker.service systemd unit (stopped + disabled).
/var/lib/docker/ Docker state (unused on current cluster).

On Cloudflare

Not a filesystem, but worth noting the dashboard hierarchy:

Websites → myhoneydue.com
├── DNS → Records              (A records for api, admin, @)
├── SSL/TLS → Overview          (SSL mode: Flexible)
├── SSL/TLS → Edge Certificates (Always Use HTTPS: On)
├── SSL/TLS → Origin Server     (would live the Origin CA cert if we enabled it)
├── Rules → Overview            (where Origin Rules live if we had them)
├── Rules → Page Rules          (none)
├── Security → WAF              (managed rules only)
├── Speed → Optimization        (default)
└── Analytics & Logs            (read-only stats)

On Gitea (gitea.treytartt.com)

The image registry lives at:

gitea.treytartt.com/admin/-/packages         # UI listing of all packages
gitea.treytartt.com/admin/-/packages/container/honeydue-api      # API image
gitea.treytartt.com/admin/-/packages/container/honeydue-worker   # Worker image
gitea.treytartt.com/admin/-/packages/container/honeydue-admin    # Admin image

Per-version tags visible in the UI with docker pull commands.

PATs at gitea.treytartt.com/-/user/settings/applications.

On Neon

console.neon.tech → project → Branches       (production branch default)
console.neon.tech → project → Monitoring     (CU-hour usage, slow queries)
console.neon.tech → project → Operations     (history of schema changes)

Connection strings at console.neon.tech → project → Connection Details.

On Backblaze B2

secure.backblaze.com/b2_buckets.htm          # Buckets list
secure.backblaze.com/b2_app_keys.htm         # App keys

honeyDueProd bucket → Files tab for browsing contents.