Mark roadmap items done (network policies, Traefik middleware, CF Full strict, CF IP UFW restriction, webapp deploy, APNs wired up, admin URL-baking fix, admin probe bug). Update Chapter 4 (firewall rule inventory now shows CF-only :443, no :80), Chapter 6 (request flow walks through TLS on :443 and middleware hops), Chapter 13 (CF SSL mode is Full strict, not Flexible; documents the origin cert install), Chapter 7 (adds the web service section — proxy pattern, 3 replicas, PostHog build-args), and Appendix C (web manifests, CF origin cert paths on disk, APNs .p8 path, updated network-policies applied status). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
11 KiB
13 — Cloudflare
Summary
Cloudflare sits in front of every public request. It provides DNS
(authoritative nameservers for myhoneydue.com), TLS termination at
the edge, DDoS mitigation, caching, and the round-robin fan-out across
our three node IPs. We use the Free plan. TLS mode is Full (strict)
— CF connects to origin over HTTPS and verifies the origin's cert
against CF's own Origin CA. This chapter documents every Cloudflare
setting that matters.
DNS
Zone
myhoneydue.com, managed by Cloudflare. Authoritative nameservers:
carol.ns.cloudflare.com
ishaan.ns.cloudflare.com
Records that matter
| Type | Name | Content | Proxy | Notes |
|---|---|---|---|---|
| A | api |
178.104.247.152 | 🟠 Proxied | hetzner1 |
| A | api |
178.105.32.198 | 🟠 Proxied | hetzner2 |
| A | api |
178.104.249.189 | 🟠 Proxied | hetzner3 |
| A | admin |
178.104.247.152 | 🟠 Proxied | same 3 IPs |
| A | admin |
178.105.32.198 | 🟠 Proxied | |
| A | admin |
178.104.249.189 | 🟠 Proxied | |
| A | @ |
178.104.247.152 | 🟠 Proxied | same 3 IPs |
| A | @ |
178.105.32.198 | 🟠 Proxied | |
| A | @ |
178.104.249.189 | 🟠 Proxied |
Three A records per name → Cloudflare selects one per request. With proxying on (orange cloud), the client never sees these IPs — it sees a Cloudflare edge IP. CF internally picks which of the three origin IPs to connect to; if one fails the connection, CF retries the next.
TXT records for email (Fastmail sending domain): SPF, DKIM, DMARC. Not our immediate concern; configured by the Fastmail custom-domain setup.
Why three A records per name, not one
With one record pointing at hetzner1:
- Only hetzner1 sees traffic
- If hetzner1 is unreachable, everything breaks until we change DNS
With three records:
- CF chooses one origin per connection
- If one node's port :80 stops responding, CF tries the others
- Node upgrades can be done one at a time with no user impact
This is poor-man's load balancing. A Hetzner Load Balancer or Cloudflare Load Balancer (paid) would be more sophisticated — with active health checks and automatic failover on sub-second latency. Our DNS approach is "good enough" for the traffic volume.
Cloudflare's origin health checks
On Free plan, CF doesn't actively probe origins. It reacts to real connection failures: if an origin returns 5xx repeatedly or connection times out, CF marks it unhealthy for that edge POP for some time.
Upgrading to Cloudflare Load Balancing ($5/mo add-on) would enable active health checks — explicit probes independent of traffic. Useful when you want sub-second failover.
TLS
Mode: Full (strict)
CF Dashboard → SSL/TLS → Overview → Full (strict).
What this means:
- User ↔ Cloudflare: TLS (HTTPS) — CF serves its own Let's Encrypt cert
- Cloudflare ↔ Origin: TLS (HTTPS :443) — origin serves our CF Origin CA cert; CF verifies it chains to CF's Origin CA root
How it's wired:
- k8s secret
cloudflare-origin-cert(typekubernetes.io/tls) holdstls.crt+tls.key. The cert is valid for*.myhoneydue.com+myhoneydue.com, 15-year validity, issued byCloudFlare Origin CA SSL Certificate Authority. - All three
Ingressresources indeploy-k3s/manifests/ingress/ingress-simple.yamlreference the secret viaspec.tls[].secretName. - Traefik terminates TLS on :443 using the cert. Backend pods still speak plain HTTP over the cluster network (Traefik → pod is an intra-cluster hop, encrypted at the Flannel overlay layer).
Why we chose Full (strict) over Flexible:
- CF → origin traffic was plaintext on Flexible. Between Cloudflare's POPs and Hetzner Nuremberg is a lot of internet. Full (strict) closes that gap.
- Origin cert is a CF-internal-only CA, so it's useless to anyone who isn't CF. Non-CF clients that somehow bypass the UFW CF-IP allowlist can't impersonate the origin because their cert wouldn't chain to CF's Origin CA root.
Maintenance: the Origin CA cert is valid for 15 years (expires
Apr 2041). No action needed until then. If rotation is ever required,
regenerate in CF dashboard → SSL/TLS → Origin Server, re-run the
kubectl create secret tls cloudflare-origin-cert --dry-run=client -o yaml | kubectl apply -f -
command, Traefik picks it up on next secret reload (no pod restart).
Regenerating the cert (for the record)
# After downloading cf-origin-cert.pem + cf-origin-key.pem from CF dashboard:
kubectl -n honeydue create secret tls cloudflare-origin-cert \
--cert=cf-origin-cert.pem \
--key=cf-origin-key.pem \
--dry-run=client -o yaml | kubectl apply -f -
Edge certificate
CF provides a free edge certificate for *.myhoneydue.com and
myhoneydue.com. Auto-renewed by Cloudflare. We don't touch it.
Always Use HTTPS
SSL/TLS → Edge Certificates → Always Use HTTPS: On (default).
Redirects any HTTP → HTTPS at the CF edge. Clients that hit
http://api.myhoneydue.com/* get 301'd to https://.... Origin never
sees the HTTP request.
HSTS
Not currently enabled. HSTS (HTTP Strict Transport Security) sends
a header telling browsers "always use HTTPS for this domain." Once set
with long max-age, it's permanent until it expires — if we later
misconfigure TLS, HSTS-enabled browsers refuse to connect at all.
Enabling HSTS is a TODO but requires confidence in our TLS stability. Not tonight.
DDoS mitigation
CF's Free plan includes basic DDoS protection:
- Volumetric attacks absorbed at the edge
- Obvious bot patterns blocked (known-bad user agents, headless browsers doing suspicious things)
Under a large attack, CF might:
- Insert a "checking your browser" JavaScript challenge (the ~5-second "Cloudflare is checking your browser" page)
- Rate-limit by IP
Under a sustained, sophisticated attack we might need:
- CF Pro plan ($20/mo) for more rule customization
- Enterprise plan for negotiated protection
- Extra measures like Cloudflare Magic Transit
So far, not needed.
Caching
Default CF caching:
- Static assets (CSS, JS, images) cached aggressively based on extension
- HTML pages honored per
Cache-Controlheaders from origin - JSON API responses typically not cached (no
Cache-Control: public)
Our Go API doesn't set Cache-Control: public on any endpoint, so CF
treats them as uncacheable. Every API call reaches origin.
If we wanted to cache certain endpoints (e.g., public lookup tables):
c.Response().Header().Set("Cache-Control", "public, max-age=300")
And CF will cache for 5 minutes.
Firewall rules at CF
CF Dashboard → Security → WAF. On Free tier:
- Managed rules: a small free allowlist of "obvious-attack" patterns
- Custom rules: limited (5 on Free, 20 on Pro)
We have no custom rules defined currently. The managed ruleset covers:
- SQL injection attempts in query strings
- Known-vulnerable bot User-Agents
- XSS attempts in common parameters
Rate limiting
CF Free: 10,000 requests per 10 minutes per IP for free rules (we haven't configured any). The API itself should have rate limits for sensitive endpoints; we don't rely on CF for that.
What CF does NOT do for us
- Authenticate users — our app does
- Authorize requests — our app does
- Encrypt pod-to-pod traffic — nothing Cloudflare can help with
- Backup origin data — CF caches but doesn't store copies persistently
Turnstile / bot management
Not enabled. If we start seeing account-creation spam, Cloudflare Turnstile (free) would be a good addition — a CAPTCHA replacement that doesn't require user interaction for most traffic.
Origin IP protection
CF proxying (orange cloud) is the primary protection of our origin IPs. When proxying is on:
- DNS queries return CF edge IPs, never origin
- HTTP/HTTPS traffic goes through CF
However, our origin IPs can leak via:
- Email sending (if the app ever sent email directly from the origin IP) — we use Fastmail so this isn't an issue
- Outbound connections (our pods connect out to Neon, B2, Fastmail from the nodes' public IPs; those IPs appear in external logs)
- Historical DNS records (services like SecurityTrails log historical DNS; if we ever had unproxied A records, attackers can look them up)
If origin IPs leak, attackers can bypass CF's protection by connecting directly to node IPs. Current mitigation:
- UFW only allows :80/:443 from anywhere
- Our app has no ports bound to the public IP
Future (Chapter 20): UFW rule to allow :80/:443 only from CF IP ranges. Prevents direct-connect bypass entirely.
Cloudflare IP ranges (used in Traefik trustedIPs)
From cloudflare.com/ips:
IPv4 ranges:
173.245.48.0/20
103.21.244.0/22
103.22.200.0/22
103.31.4.0/22
141.101.64.0/18
108.162.192.0/18
190.93.240.0/20
188.114.96.0/20
197.234.240.0/22
198.41.128.0/17
162.158.0.0/15
104.16.0.0/13
104.24.0.0/14
172.64.0.0/13
131.0.72.0/22
IPv6 ranges:
2400:cb00::/32
2606:4700::/32
2803:f800::/32
2405:b500::/32
2405:8100::/32
2a06:98c0::/29
2c0f:f248::/32
These are used in two places:
- Traefik
forwardedHeaders.trustedIPs— we already have this configured (Chapter 6) - UFW
allow 80/tcp from <cf-range>— NOT configured (TODO)
CF occasionally adds new ranges. If a future CF range isn't in our list, we'd either trust unknown IPs (if lax) or reject legitimate CF traffic (if strict). The canonical source is the public API:
curl -sS https://www.cloudflare.com/ips-v4
curl -sS https://www.cloudflare.com/ips-v6
API token for programmatic changes
If we automate DNS changes (e.g., adding new subdomain on deploy),
we'd need a CF API token with Zone:DNS:Edit scope for the
myhoneydue.com zone.
Currently not automated; DNS is managed in the CF dashboard by hand.
Cost
$0/mo. Free plan covers everything we use. Paid plans add features we don't need yet:
| Feature | Free | Pro ($20) | Business ($200) |
|---|---|---|---|
| DNS + proxying | ✓ | ✓ | ✓ |
| Basic DDoS | ✓ | ✓ | ✓ |
| SSL (edge + Flexible + Full + Full strict) | ✓ | ✓ | ✓ |
| WAF managed rules | ✓ (limited) | ✓ (more) | ✓ (all) |
| Custom firewall rules | 5 | 20 | 100 |
| Page Rules | 3 | 20 | 50 |
| Image Resizing | no | no | ✓ |
| Load Balancing | no | $5/mo add-on | ✓ |
We'd consider Pro ($20/mo) if:
- We needed a custom WAF rule beyond the 5-rule limit
- We wanted Image Resizing for user-uploaded photos
Neither is needed today.
Operator cheat sheet
# Query current CF-served DNS
dig +short @1.1.1.1 api.myhoneydue.com # returns CF edge IPs when proxied
# Query our origin directly (bypass CF)
curl -sS -H "Host: api.myhoneydue.com" http://178.104.247.152/api/health/
# Check CF headers (confirm you're going through CF)
curl -sS -I https://api.myhoneydue.com/api/health/ | grep -i cf-
# Purge CF cache (requires API token)
curl -X POST \
-H "Authorization: Bearer $CF_TOKEN" \
-H "Content-Type: application/json" \
"https://api.cloudflare.com/client/v4/zones/<zone_id>/purge_cache" \
-d '{"purge_everything":true}'