Files
Trey t 7e77e3bbab
Backend CI / Test (push) Has been cancelled
Backend CI / Contract Tests (push) Has been cancelled
Backend CI / Build (push) Has been cancelled
Backend CI / Lint (push) Has been cancelled
Backend CI / Secret Scanning (push) Has been cancelled
docs/deployment: record security hardening pass + webapp + APNs
Mark roadmap items done (network policies, Traefik middleware, CF Full
strict, CF IP UFW restriction, webapp deploy, APNs wired up, admin
URL-baking fix, admin probe bug). Update Chapter 4 (firewall rule
inventory now shows CF-only :443, no :80), Chapter 6 (request flow
walks through TLS on :443 and middleware hops), Chapter 13 (CF SSL
mode is Full strict, not Flexible; documents the origin cert
install), Chapter 7 (adds the web service section — proxy pattern,
3 replicas, PostHog build-args), and Appendix C (web manifests, CF
origin cert paths on disk, APNs .p8 path, updated network-policies
applied status).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 15:50:59 -05:00

342 lines
11 KiB
Markdown

# 13 — Cloudflare
## Summary
Cloudflare sits in front of every public request. It provides DNS
(authoritative nameservers for `myhoneydue.com`), TLS termination at
the edge, DDoS mitigation, caching, and the round-robin fan-out across
our three node IPs. We use the Free plan. TLS mode is **Full (strict)**
— CF connects to origin over HTTPS and verifies the origin's cert
against CF's own Origin CA. This chapter documents every Cloudflare
setting that matters.
## DNS
### Zone
`myhoneydue.com`, managed by Cloudflare. Authoritative nameservers:
```
carol.ns.cloudflare.com
ishaan.ns.cloudflare.com
```
### Records that matter
| Type | Name | Content | Proxy | Notes |
|---|---|---|---|---|
| A | `api` | 178.104.247.152 | 🟠 Proxied | hetzner1 |
| A | `api` | 178.105.32.198 | 🟠 Proxied | hetzner2 |
| A | `api` | 178.104.249.189 | 🟠 Proxied | hetzner3 |
| A | `admin` | 178.104.247.152 | 🟠 Proxied | same 3 IPs |
| A | `admin` | 178.105.32.198 | 🟠 Proxied | |
| A | `admin` | 178.104.249.189 | 🟠 Proxied | |
| A | `@` | 178.104.247.152 | 🟠 Proxied | same 3 IPs |
| A | `@` | 178.105.32.198 | 🟠 Proxied | |
| A | `@` | 178.104.249.189 | 🟠 Proxied | |
Three A records per name → Cloudflare selects one per request. With
proxying on (orange cloud), **the client never sees these IPs** — it
sees a Cloudflare edge IP. CF internally picks which of the three
origin IPs to connect to; if one fails the connection, CF retries the
next.
**TXT records for email** (Fastmail sending domain): SPF, DKIM, DMARC.
Not our immediate concern; configured by the Fastmail custom-domain
setup.
### Why three A records per name, not one
With one record pointing at hetzner1:
- Only hetzner1 sees traffic
- If hetzner1 is unreachable, everything breaks until we change DNS
With three records:
- CF chooses one origin per connection
- If one node's port :80 stops responding, CF tries the others
- Node upgrades can be done one at a time with no user impact
This is poor-man's load balancing. A Hetzner Load Balancer or Cloudflare
Load Balancer (paid) would be more sophisticated — with active health
checks and automatic failover on sub-second latency. Our DNS approach
is "good enough" for the traffic volume.
### Cloudflare's origin health checks
On Free plan, CF doesn't actively probe origins. It reacts to real
connection failures: if an origin returns 5xx repeatedly or connection
times out, CF marks it unhealthy for that edge POP for some time.
Upgrading to **Cloudflare Load Balancing** ($5/mo add-on) would enable
active health checks — explicit probes independent of traffic. Useful
when you want sub-second failover.
## TLS
### Mode: Full (strict)
CF Dashboard → SSL/TLS → Overview → **Full (strict)**.
**What this means:**
- User ↔ Cloudflare: **TLS** (HTTPS) — CF serves its own Let's Encrypt cert
- Cloudflare ↔ Origin: **TLS** (HTTPS :443) — origin serves our CF Origin CA cert; CF verifies it chains to CF's Origin CA root
**How it's wired:**
- k8s secret `cloudflare-origin-cert` (type `kubernetes.io/tls`) holds
`tls.crt` + `tls.key`. The cert is valid for `*.myhoneydue.com` +
`myhoneydue.com`, 15-year validity, issued by
`CloudFlare Origin CA SSL Certificate Authority`.
- All three `Ingress` resources in `deploy-k3s/manifests/ingress/ingress-simple.yaml`
reference the secret via `spec.tls[].secretName`.
- Traefik terminates TLS on :443 using the cert. Backend pods still
speak plain HTTP over the cluster network (Traefik → pod is an
intra-cluster hop, encrypted at the Flannel overlay layer).
**Why we chose Full (strict) over Flexible:**
- CF → origin traffic was plaintext on Flexible. Between Cloudflare's
POPs and Hetzner Nuremberg is a lot of internet. Full (strict)
closes that gap.
- Origin cert is a CF-internal-only CA, so it's useless to anyone who
isn't CF. Non-CF clients that somehow bypass the UFW CF-IP allowlist
can't impersonate the origin because their cert wouldn't chain to
CF's Origin CA root.
**Maintenance:** the Origin CA cert is valid for 15 years (expires
Apr 2041). No action needed until then. If rotation is ever required,
regenerate in CF dashboard → SSL/TLS → Origin Server, re-run the
`kubectl create secret tls cloudflare-origin-cert --dry-run=client -o yaml | kubectl apply -f -`
command, Traefik picks it up on next secret reload (no pod restart).
### Regenerating the cert (for the record)
```bash
# After downloading cf-origin-cert.pem + cf-origin-key.pem from CF dashboard:
kubectl -n honeydue create secret tls cloudflare-origin-cert \
--cert=cf-origin-cert.pem \
--key=cf-origin-key.pem \
--dry-run=client -o yaml | kubectl apply -f -
```
### Edge certificate
CF provides a free edge certificate for `*.myhoneydue.com` and
`myhoneydue.com`. Auto-renewed by Cloudflare. We don't touch it.
### Always Use HTTPS
SSL/TLS → Edge Certificates → **Always Use HTTPS: On** (default).
Redirects any HTTP → HTTPS at the CF edge. Clients that hit
`http://api.myhoneydue.com/*` get 301'd to `https://...`. Origin never
sees the HTTP request.
### HSTS
**Not currently enabled.** HSTS (HTTP Strict Transport Security) sends
a header telling browsers "always use HTTPS for this domain." Once set
with long `max-age`, it's **permanent** until it expires — if we later
misconfigure TLS, HSTS-enabled browsers refuse to connect at all.
Enabling HSTS is a TODO but requires confidence in our TLS stability.
Not tonight.
## DDoS mitigation
CF's Free plan includes basic DDoS protection:
- Volumetric attacks absorbed at the edge
- Obvious bot patterns blocked (known-bad user agents, headless browsers
doing suspicious things)
Under a large attack, CF might:
- Insert a "checking your browser" JavaScript challenge (the ~5-second
"Cloudflare is checking your browser" page)
- Rate-limit by IP
Under a sustained, sophisticated attack we might need:
- CF Pro plan ($20/mo) for more rule customization
- Enterprise plan for negotiated protection
- Extra measures like Cloudflare Magic Transit
So far, not needed.
## Caching
Default CF caching:
- Static assets (CSS, JS, images) cached aggressively based on extension
- HTML pages honored per `Cache-Control` headers from origin
- JSON API responses typically not cached (no `Cache-Control: public`)
Our Go API doesn't set `Cache-Control: public` on any endpoint, so CF
treats them as uncacheable. Every API call reaches origin.
If we wanted to cache certain endpoints (e.g., public lookup tables):
```go
c.Response().Header().Set("Cache-Control", "public, max-age=300")
```
And CF will cache for 5 minutes.
## Firewall rules at CF
CF Dashboard → Security → WAF. On Free tier:
- Managed rules: a small free allowlist of "obvious-attack" patterns
- Custom rules: limited (5 on Free, 20 on Pro)
We have **no custom rules defined** currently. The managed ruleset
covers:
- SQL injection attempts in query strings
- Known-vulnerable bot User-Agents
- XSS attempts in common parameters
## Rate limiting
CF Free: **10,000 requests per 10 minutes per IP for free rules** (we
haven't configured any). The API itself should have rate limits for
sensitive endpoints; we don't rely on CF for that.
## What CF does NOT do for us
- **Authenticate users** — our app does
- **Authorize requests** — our app does
- **Encrypt pod-to-pod traffic** — nothing Cloudflare can help with
- **Backup origin data** — CF caches but doesn't store copies
persistently
## Turnstile / bot management
Not enabled. If we start seeing account-creation spam, Cloudflare
Turnstile (free) would be a good addition — a CAPTCHA replacement that
doesn't require user interaction for most traffic.
## Origin IP protection
CF proxying (orange cloud) is the primary protection of our origin IPs.
When proxying is on:
- DNS queries return CF edge IPs, never origin
- HTTP/HTTPS traffic goes through CF
However, our origin IPs **can leak** via:
- Email sending (if the app ever sent email directly from the origin IP)
— we use Fastmail so this isn't an issue
- Outbound connections (our pods connect out to Neon, B2, Fastmail from
the nodes' public IPs; those IPs appear in external logs)
- Historical DNS records (services like SecurityTrails log historical
DNS; if we ever had unproxied A records, attackers can look them up)
**If origin IPs leak**, attackers can bypass CF's protection by
connecting directly to node IPs. Current mitigation:
- UFW only allows :80/:443 from anywhere
- Our app has no ports bound to the public IP
**Future** (Chapter 20): UFW rule to allow :80/:443 only from CF IP
ranges. Prevents direct-connect bypass entirely.
## Cloudflare IP ranges (used in Traefik trustedIPs)
From [cloudflare.com/ips](https://www.cloudflare.com/ips/):
IPv4 ranges:
```
173.245.48.0/20
103.21.244.0/22
103.22.200.0/22
103.31.4.0/22
141.101.64.0/18
108.162.192.0/18
190.93.240.0/20
188.114.96.0/20
197.234.240.0/22
198.41.128.0/17
162.158.0.0/15
104.16.0.0/13
104.24.0.0/14
172.64.0.0/13
131.0.72.0/22
```
IPv6 ranges:
```
2400:cb00::/32
2606:4700::/32
2803:f800::/32
2405:b500::/32
2405:8100::/32
2a06:98c0::/29
2c0f:f248::/32
```
These are used in two places:
1. **Traefik `forwardedHeaders.trustedIPs`** — we already have this
configured (Chapter 6)
2. **UFW `allow 80/tcp from <cf-range>`** — NOT configured (TODO)
CF occasionally adds new ranges. If a future CF range isn't in our
list, we'd either trust unknown IPs (if lax) or reject legitimate CF
traffic (if strict). The canonical source is the public API:
```bash
curl -sS https://www.cloudflare.com/ips-v4
curl -sS https://www.cloudflare.com/ips-v6
```
## API token for programmatic changes
If we automate DNS changes (e.g., adding new subdomain on deploy),
we'd need a CF API token with `Zone:DNS:Edit` scope for the
`myhoneydue.com` zone.
Currently not automated; DNS is managed in the CF dashboard by hand.
## Cost
**$0/mo**. Free plan covers everything we use. Paid plans add features
we don't need yet:
| Feature | Free | Pro ($20) | Business ($200) |
|---|---|---|---|
| DNS + proxying | ✓ | ✓ | ✓ |
| Basic DDoS | ✓ | ✓ | ✓ |
| SSL (edge + Flexible + Full + Full strict) | ✓ | ✓ | ✓ |
| WAF managed rules | ✓ (limited) | ✓ (more) | ✓ (all) |
| Custom firewall rules | 5 | 20 | 100 |
| Page Rules | 3 | 20 | 50 |
| Image Resizing | no | no | ✓ |
| Load Balancing | no | $5/mo add-on | ✓ |
We'd consider Pro ($20/mo) if:
- We needed a custom WAF rule beyond the 5-rule limit
- We wanted Image Resizing for user-uploaded photos
Neither is needed today.
## Operator cheat sheet
```bash
# Query current CF-served DNS
dig +short @1.1.1.1 api.myhoneydue.com # returns CF edge IPs when proxied
# Query our origin directly (bypass CF)
curl -sS -H "Host: api.myhoneydue.com" http://178.104.247.152/api/health/
# Check CF headers (confirm you're going through CF)
curl -sS -I https://api.myhoneydue.com/api/health/ | grep -i cf-
# Purge CF cache (requires API token)
curl -X POST \
-H "Authorization: Bearer $CF_TOKEN" \
-H "Content-Type: application/json" \
"https://api.cloudflare.com/client/v4/zones/<zone_id>/purge_cache" \
-d '{"purge_everything":true}'
```
## References
- [Cloudflare IP ranges][cf-ips]
- [Cloudflare SSL modes explained][cf-ssl]
- [Origin CA certificates][cf-origin-ca]
- [Cloudflare DNS best practices][cf-dns]
[cf-ips]: https://www.cloudflare.com/ips/
[cf-ssl]: https://developers.cloudflare.com/ssl/origin-configuration/ssl-modes/
[cf-origin-ca]: https://developers.cloudflare.com/ssl/origin-configuration/origin-ca/
[cf-dns]: https://developers.cloudflare.com/dns/