Migrate prod deploy from Swarm to K3s; add full deployment book
Infrastructure:
- Stack now runs on K3s v1.34.6 HA (3 Hetzner CX33 nodes as managers)
- Traefik DaemonSet + hostNetwork replaces Caddy + ingress mesh
- All manifests in deploy-k3s/manifests/; Swarm config (deploy/) kept
temporarily for reference
Bug fixes surfaced during migration:
- Dockerfile: golang:1.24-alpine -> 1.25-alpine (go.mod requires 1.25)
- cache_service.go: remove sync.Once reassignment from inside Do()
callback (was causing 'unlock of unlocked mutex' fatal after
Redis Ping failure)
- router.go: relax CSP from 'default-src none' to 'default-src self'
+ allowlist fonts.googleapis.com so the marketing landing page CSS
actually loads in browsers
- deploy/scripts/deploy_prod.sh: use docker buildx with
--platform linux/amd64 so arm64 (Apple Silicon) dev machines produce
images runnable on x86_64 Hetzner nodes; fix array expansion under
set -u
- deploy/swarm-stack.prod.yml: fix secret source references to use
top-level aliases (the '\${X_SECRET}' form never actually resolved);
dozzle ports: long-form host_ip is rejected by Swarm, switched to
short-form (bound to 0.0.0.0 with UFW-based loopback restriction);
worker replicas 2 -> 1 (Asynq scheduler singleton)
- deploy-k3s/manifests/admin/deployment.yaml: probe path '/admin/' -> '/'
(Next.js serves at root; /admin/ returned 404 and killed pods);
startupProbe failureThreshold 12 -> 24
- deploy-k3s/manifests/pod-disruption-budgets.yaml: worker minAvailable
1 -> 0 (singleton)
- deploy-k3s/manifests/api/deployment.yaml: startupProbe failureThreshold
12 -> 48 (MigrateWithLock serializes across 3 replicas on first-boot;
real startup takes up to 240s)
- .gitignore: tighten 'api' -> '/api' (was matching deploy-k3s/manifests/api/
and admin/src/app/api/*, hiding legitimate files)
New files:
- deploy-k3s/manifests/traefik-helmchartconfig.yaml: DaemonSet +
hostNetwork override for k3s-bundled Traefik
- deploy-k3s/manifests/ingress/ingress-simple.yaml: plain Ingress
without TLS (CF Flexible SSL) and without middleware
- deploy-k3s/MIGRATION_NOTES.md: operator-facing migration log
Documentation:
- docs/deployment/ — full deployment book, 26 files, ~42k words:
- Part I Overview, infrastructure, orchestrator choice (Ch 0-2)
- Part II Networking, firewall, Cloudflare (Ch 3-4, 13)
- Part III Security, Traefik ingress (Ch 5-6)
- Part IV Services, DB, storage, secrets, registry (Ch 7-11)
- Part V Data flow, deploy process, observability, failures, runbook
(Ch 12, 14-17)
- Part VI Cost, Swarm postmortem, roadmap (Ch 18-20)
- Appendices: glossary, kubectl cheat sheet, file locations,
consolidated citations
- README.md: Production Deployment section replaced with pointer to
the book; Go version bumped to 1.25
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,52 @@
|
||||
# honeyDue edge proxy — terminates HTTP from Cloudflare, routes by Host header.
|
||||
#
|
||||
# Cloudflare is in front, SSL mode "Flexible" — CF terminates TLS at the edge
|
||||
# and talks to this origin over plain HTTP on port 80. No LE certs needed here
|
||||
# for now. Later, to go "Full (strict)", remove `auto_https off`, add `tls` blocks
|
||||
# that use the ACME HTTP-01 challenge, and open 443 on the node.
|
||||
|
||||
{
|
||||
admin off
|
||||
auto_https off
|
||||
}
|
||||
|
||||
# api.myhoneydue.com → Go REST API
|
||||
# `dynamic a` re-resolves the Swarm service DNS every 30s instead of caching
|
||||
# the IP forever at config parse. This is critical on Swarm with endpoint_mode:
|
||||
# dnsrr — when a task restarts, its overlay IP changes, and static DNS caching
|
||||
# leaves Caddy dialing dead IPs.
|
||||
api.myhoneydue.com:80 {
|
||||
reverse_proxy {
|
||||
dynamic a {
|
||||
name api
|
||||
port 8000
|
||||
refresh 30s
|
||||
}
|
||||
header_up X-Forwarded-Proto {http.request.header.X-Forwarded-Proto}
|
||||
}
|
||||
}
|
||||
|
||||
# admin.myhoneydue.com → Next.js admin panel via overlay DNS (VIP endpoint)
|
||||
#
|
||||
# This relies on Swarm's embedded resolver, which has a known libnetwork
|
||||
# stale-record bug (moby/moby#52265, affects 29.x). We work around it by
|
||||
# (a) using default VIP endpoint_mode — a stable service IP — and
|
||||
# (b) running a clean overlay from scratch (see Phase 1 stack recreate).
|
||||
#
|
||||
# If ghosts come back, the long-term fix is Traefik w/ Swarm provider that
|
||||
# reads task IPs from Docker API, bypassing libnetwork DNS entirely. See
|
||||
# deploy/MIGRATION_NOTES.md for the Traefik migration plan.
|
||||
admin.myhoneydue.com:80 {
|
||||
reverse_proxy admin:3000 {
|
||||
lb_try_duration 3s
|
||||
lb_try_interval 250ms
|
||||
header_up X-Forwarded-Proto {http.request.header.X-Forwarded-Proto}
|
||||
}
|
||||
}
|
||||
|
||||
# Catch-all for root/unknown hostnames hitting our IPs directly.
|
||||
# Cloudflare SSL=Flexible will still hit us on :80 for myhoneydue.com; return
|
||||
# a placeholder until you wire a real marketing site.
|
||||
:80 {
|
||||
respond "honeyDue" 200
|
||||
}
|
||||
Reference in New Issue
Block a user