fix(security): remediate 2026-05-12 audit findings (Stages 2–5)

Remediation of the 2026-05-12/13 audits (78 findings + cluster gaps),
tracked in deploy-k3s/SECURITY.md, plus fixes from two independent
post-remediation reviews.

Auth & sessions:
- SHA-256 hashed auth-token storage (C1); prior-token cache eviction on
  re-login (MEDIUM-1)
- local Google JWKS verification, iss/aud/exp checks (C2/C3)
- constant-time login + generic errors (L1/LIVE-L11/LIVE-L13)
- per-account login lockout keyed on distinct source IPs (M5/MEDIUM-3)
- verified-email gating, login rate limiting (LIVE-L19, H1-H3)

IAP & webhooks:
- Apple/Google cross-account replay protection (C5/C6/C10/C13, H5/H6)
- migrations 000003-000006 (token hashing, IAP replay, audit_log +
  webhook_event_log table creation, append-only audit log)

Authorization & races:
- file-ownership owner-OR-member fix (C7), atomic share-code join
  (C9/H9), device-token reassignment (C8/LOW-3)

Secrets & deploy:
- secrets file-mounted at /etc/honeydue/secrets, not env (F8); Redis
  password out of the ConfigMap (HIGH-1); B2 keys reconciled
- digest-pinned images, admin ingress hardening, CSP/HSTS, /metrics
  lockdown; kubeconfig 0600, etcd secrets-encryption, fail2ban +
  unattended-upgrades at provision; secret-rotation runbook

Build, vet, and the full test suite (incl. -race) pass; the goose
migration chain is verified against PostgreSQL 16.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

deploy-k3s/manifests/admin/deployment.yaml

@@ -23,8 +23,11 @@ spec:
         app.kubernetes.io/part-of: honeydue
     spec:
       serviceAccountName: admin
+      # Explicit pod-level opt-out (audit F11) — defense-in-depth on top of
+      # the ServiceAccount-level setting in rbac.yaml.
+      automountServiceAccountToken: false
       imagePullSecrets:
-        - name: ghcr-credentials
+        - name: gitea-credentials
       securityContext:
         runAsNonRoot: true
         runAsUser: 1001
@@ -35,6 +38,7 @@ spec:
       containers:
         - name: admin
           image: IMAGE_PLACEHOLDER  # Replaced by 03-deploy.sh
+          imagePullPolicy: IfNotPresent  # audit CODE-L4 — explicit; images are SHA/digest-pinned
           ports:
             - containerPort: 3000
               protocol: TCP

deploy-k3s/manifests/api/deployment.yaml

@@ -23,8 +23,11 @@ spec:
         app.kubernetes.io/part-of: honeydue
     spec:
       serviceAccountName: api
+      # Explicit pod-level opt-out (audit F11) — defense-in-depth on top of
+      # the ServiceAccount-level setting in rbac.yaml.
+      automountServiceAccountToken: false
       imagePullSecrets:
-        - name: ghcr-credentials
+        - name: gitea-credentials
       securityContext:
         runAsNonRoot: true
         runAsUser: 1000
@@ -35,6 +38,7 @@ spec:
       containers:
         - name: api
           image: IMAGE_PLACEHOLDER  # Replaced by 03-deploy.sh
+          imagePullPolicy: IfNotPresent  # audit CODE-L4 — explicit; images are SHA/digest-pinned
           ports:
             - containerPort: 8000
               protocol: TCP
@@ -46,65 +50,16 @@ spec:
           envFrom:
             - configMapRef:
                 name: honeydue-config
-          env:
-            - name: POSTGRES_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: POSTGRES_PASSWORD
-            - name: SECRET_KEY
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: SECRET_KEY
-            - name: EMAIL_HOST_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: EMAIL_HOST_PASSWORD
-            - name: FCM_SERVER_KEY
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: FCM_SERVER_KEY
-            - name: REDIS_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: REDIS_PASSWORD
-                  optional: true
-            # B2 (Backblaze) credentials. With both set, StorageConfig.IsS3()
-            # returns true and uploads stream to B2 via minio-go. With either
-            # missing, code falls back to local filesystem — and since
-            # readOnlyRootFilesystem is true on this container, that fallback
-            # silently fails. So both must be wired or uploads break.
-            - name: B2_KEY_ID
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: B2_KEY_ID
-            - name: B2_APP_KEY
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: B2_APP_KEY
-            # Observability — push traces (and any future OTLP metrics) to
-            # obs.88oakapps.com. Token gates ingest at nginx; URL is the
-            # same one vmagent uses for metric remote-write. Both come from
-            # honeydue-secrets so they aren't world-readable in ConfigMap.
-            - name: OBS_TRACES_URL
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: OBS_TRACES_URL
-                  optional: true
-            - name: OBS_INGEST_TOKEN
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: OBS_INGEST_TOKEN
-                  optional: true
+          # Audit CODE-F8: secrets are NOT injected as environment variables.
+          # Env vars are readable for the life of the pod via /proc/<pid>/environ
+          # and leak into crash dumps / child processes. honeydue-secrets is
+          # mounted read-only at /etc/honeydue/secrets (mode 0400) and the Go
+          # config layer (config.loadFileSecrets) reads each key from its file.
+          # Non-secret config still arrives via the configMapRef above.
           volumeMounts:
+            - name: app-secrets
+              mountPath: /etc/honeydue/secrets
+              readOnly: true
             - name: apns-key
               mountPath: /secrets/apns
               readOnly: true
@@ -121,11 +76,12 @@ spec:
             httpGet:
               path: /api/health/
               port: 8000
-            # MigrateWithLock in cmd/api/main.go runs pg_advisory_lock on
-            # every startup. On a cold boot with 3 replicas, the first does
-            # AutoMigrate (~90s) and the others wait on the lock, so real
-            # startup runs 90–240s. 48 × 5s = 240s grace absorbs it without
-            # healthcheck killing a still-starting replica.
+            # Schema migrations run separately in the honeydue-migrate Job
+            # *before* this Deployment rolls — the api itself does not migrate
+            # (it only verifies goose_db_version at boot). Cold start still
+            # pays the DB pool warm-up + Redis connect + APNs/FCM client init
+            # before /api/health/ goes green. 48 × 5s = 240s grace keeps the
+            # probe from killing a still-starting replica.
             failureThreshold: 48
             periodSeconds: 5
           readinessProbe:
@@ -143,6 +99,12 @@ spec:
             periodSeconds: 30
             timeoutSeconds: 10
       volumes:
+        # Audit CODE-F8: the whole honeydue-secrets Secret, projected as files.
+        # defaultMode 0400 → readable only by the container's runAsUser (1000).
+        - name: app-secrets
+          secret:
+            secretName: honeydue-secrets
+            defaultMode: 0400
         - name: apns-key
           secret:
             secretName: honeydue-apns-key

deploy-k3s/manifests/ingress/ingress-simple.yaml

@@ -53,7 +53,12 @@ metadata:
   labels:
     app.kubernetes.io/part-of: honeydue
   annotations:
-    traefik.ingress.kubernetes.io/router.middlewares: honeydue-security-headers@kubernetescrd,honeydue-rate-limit@kubernetescrd
+    # cloudflare-only + admin-auth wired in (audit F2/F3/CODE-L6). Order
+    # matters: reject non-Cloudflare IPs, then basic auth, then headers,
+    # then rate limit. The admin-basic-auth secret is created by
+    # 02-setup-secrets.sh from config.yaml admin.basic_auth_* — that runs
+    # before 03-deploy.sh, so the middleware always has its secret.
+    traefik.ingress.kubernetes.io/router.middlewares: honeydue-cloudflare-only@kubernetescrd,honeydue-admin-auth@kubernetescrd,honeydue-security-headers@kubernetescrd,honeydue-rate-limit@kubernetescrd
 spec:
   ingressClassName: traefik
   tls:
@@ -98,3 +103,98 @@ spec:
                 name: web
                 port:
                   number: 3000
+---
+# Auth-endpoint Ingress (audit F10 / LIVE-L12). A dedicated Ingress for the
+# auth paths so Traefik gives their longer path-prefix routers a higher
+# priority than honeydue-api's "/" router — these paths then get
+# auth-rate-limit (5/min) instead of the general rate-limit (100/min).
+# Anything not matched here falls through to honeydue-api unchanged.
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: honeydue-api-auth
+  namespace: honeydue
+  labels:
+    app.kubernetes.io/part-of: honeydue
+  annotations:
+    traefik.ingress.kubernetes.io/router.middlewares: honeydue-auth-rate-limit@kubernetescrd,honeydue-security-headers@kubernetescrd
+spec:
+  ingressClassName: traefik
+  tls:
+    - hosts:
+        - api.myhoneydue.com
+      secretName: cloudflare-origin-cert
+  rules:
+    - host: api.myhoneydue.com
+      http:
+        paths:
+          - path: /api/auth/login
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000
+          - path: /api/auth/register
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000
+          - path: /api/auth/forgot-password
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000
+          - path: /api/auth/reset-password
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000
+          - path: /api/residences/join-with-code
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000
+          - path: /api/auth/verify-reset-code
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000
+          - path: /api/auth/apple-sign-in
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000
+          - path: /api/auth/google-sign-in
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000
+          - path: /api/auth/refresh
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000
+          - path: /api/auth/account
+            pathType: Prefix
+            backend:
+              service:
+                name: api
+                port:
+                  number: 8000

deploy-k3s/manifests/ingress/middleware.yaml

@@ -21,12 +21,20 @@ spec:
   headers:
     frameDeny: true
     contentTypeNosniff: true
-    browserXssFilter: true
+    # browserXssFilter removed (audit L7): it emits the deprecated
+    # X-XSS-Protection header, which can itself introduce XSS in legacy
+    # browsers. Modern browsers ignore it.
     referrerPolicy: "strict-origin-when-cross-origin"
     customResponseHeaders:
       X-Content-Type-Options: "nosniff"
       X-Frame-Options: "DENY"
-      Strict-Transport-Security: "max-age=31536000; includeSubDomains"
+      # HSTS: 2-year max-age + preload (audit L5/CODE-L3). After this is
+      # live on api/admin/app, submit myhoneydue.com to hstspreload.org.
+      Strict-Transport-Security: "max-age=63072000; includeSubDomains; preload"
+      # Cross-origin isolation (audit F9). COEP (require-corp) is omitted —
+      # it commonly breaks third-party embeds; add only after testing.
+      Cross-Origin-Opener-Policy: "same-origin"
+      Cross-Origin-Resource-Policy: "same-origin"
       # Content-Security-Policy is intentionally NOT set here — the Go API
       # sets a CSP in internal/router/router.go that permits Google Fonts
       # for the landing page. Two CSP headers would intersect and break it.
@@ -83,3 +91,24 @@ spec:
   basicAuth:
     secret: admin-basic-auth
     realm: "honeyDue Admin"
+
+---
+# Strict rate limit for auth endpoints (audit F10 / LIVE-L12).
+# Applied via the honeydue-api-auth Ingress to login / register /
+# forgot-password / reset-password / join-with-code. depth: 2 makes the
+# limiter key on the real client IP rather than the Cloudflare edge IP
+# (request path: client -> Cloudflare -> Traefik). This is the edge half;
+# the per-account lockout in the Go app is the robust half.
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: auth-rate-limit
+  namespace: honeydue
+spec:
+  rateLimit:
+    average: 5
+    burst: 10
+    period: 1m
+    sourceCriterion:
+      ipStrategy:
+        depth: 2

deploy-k3s/manifests/kyverno-verify-images.yaml

@@ -0,0 +1,61 @@
+# Kyverno image-signature verification policy (audit CODE-L5).
+#
+# ──────────────────────────────────────────────────────────────────────────
+# THIS MANIFEST IS NOT APPLIED BY 03-deploy.sh. It is intentionally outside
+# the script's apply set. Applying it before the prerequisites are in place
+# would block every honeydue Pod from scheduling. Operator steps:
+#
+#   1. Install Kyverno in the cluster (it is an admission controller):
+#        kubectl create -f https://github.com/kyverno/kyverno/releases/latest/download/install.yaml
+#   2. Generate a cosign key pair and keep the private key safe:
+#        cosign generate-key-pair                 # -> cosign.key (PRIVATE) + cosign.pub
+#      Set COSIGN_KEY=cosign.key in the deploy environment so 03-deploy.sh
+#      signs images after pushing them (the signing step is already wired,
+#      guarded, into 03-deploy.sh).
+#   3. Paste the contents of cosign.pub into the publicKeys block below.
+#   4. Apply this policy:  kubectl apply -f deploy-k3s/manifests/kyverno-verify-images.yaml
+#   5. After confirming honeydue Pods still schedule, flip
+#      validationFailureAction from Audit to Enforce.
+#
+# Until then it is a documented, ready-to-use template — not active config.
+# ──────────────────────────────────────────────────────────────────────────
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: verify-honeydue-images
+  annotations:
+    policies.kyverno.io/title: Verify honeyDue image signatures
+    policies.kyverno.io/description: >-
+      Requires that honeyDue application images pulled into the honeydue
+      namespace carry a valid cosign signature made with the operator's key.
+spec:
+  # Audit first — logs violations without blocking. Switch to Enforce once
+  # signing is confirmed working end to end.
+  validationFailureAction: Audit
+  background: false
+  webhookTimeoutSeconds: 30
+  rules:
+    - name: verify-gitea-image-signatures
+      match:
+        any:
+          - resources:
+              kinds:
+                - Pod
+              namespaces:
+                - honeydue
+      verifyImages:
+        # Only the images we build and sign. Public base images
+        # (redis, vmagent) are pinned by digest instead — see their manifests.
+        - imageReferences:
+            - "gitea.treytartt.com/admin/honeydue-api*"
+            - "gitea.treytartt.com/admin/honeydue-worker*"
+            - "gitea.treytartt.com/admin/honeydue-admin*"
+            - "gitea.treytartt.com/admin/honeydue-web*"
+          attestors:
+            - count: 1
+              entries:
+                - keys:
+                    publicKeys: |-
+                      -----BEGIN PUBLIC KEY-----
+                      REPLACE_WITH_CONTENTS_OF_cosign.pub
+                      -----END PUBLIC KEY-----

deploy-k3s/manifests/migrate/job.yaml

@@ -27,8 +27,10 @@ spec:
         app.kubernetes.io/part-of: honeydue
     spec:
       restartPolicy: Never
+      # The migrate Job never calls the k8s API (audit F11).
+      automountServiceAccountToken: false
       imagePullSecrets:
-        - name: ghcr-credentials
+        - name: gitea-credentials
       securityContext:
         runAsNonRoot: true
         runAsUser: 1000
@@ -38,6 +40,7 @@ spec:
       containers:
         - name: goose
           image: IMAGE_PLACEHOLDER  # Replaced by 03-deploy.sh — same as api
+          imagePullPolicy: IfNotPresent  # audit CODE-L4 — explicit
           command: ["/bin/sh", "-c"]
           # DB_HOST in the ConfigMap points at the -pooler endpoint for runtime.
           # goose's session-scoped advisory lock can't survive PgBouncer

deploy-k3s/manifests/observability/vmagent.yaml

@@ -179,7 +179,17 @@ spec:
           type: RuntimeDefault
       containers:
         - name: vmagent
-          image: victoriametrics/vmagent:v1.106.1
+          # Pinned by digest (audit K3S-F14).
+          image: victoriametrics/vmagent:v1.106.1@sha256:90208a667c0baf65f7536b92a84c40b6e35ffe8e88bda7e4447b97b06c6ba6b8
+          imagePullPolicy: IfNotPresent  # audit CODE-L4 — explicit
+          # Container-level hardening (audit F7) — matches the other 5
+          # workloads. vmagent only writes to the /tmp/vmagent emptyDir
+          # (its remoteWrite buffer), so a read-only root filesystem holds.
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            capabilities:
+              drop: ["ALL"]
           args:
             - "-promscrape.config=/etc/vmagent/scrape.yaml"
             - "-remoteWrite.url=https://obs.88oakapps.com/api/v1/write"

deploy-k3s/manifests/redis/deployment.yaml

@@ -20,6 +20,9 @@ spec:
         app.kubernetes.io/part-of: honeydue
     spec:
       serviceAccountName: redis
+      # Explicit pod-level opt-out (audit F11) — defense-in-depth on top of
+      # the ServiceAccount-level setting in rbac.yaml.
+      automountServiceAccountToken: false
       nodeSelector:
         honeydue/redis: "true"
       securityContext:
@@ -31,7 +34,9 @@ spec:
           type: RuntimeDefault
       containers:
         - name: redis
-          image: redis:7-alpine
+          # Pinned by digest (audit K3S-F14) — redis:7-alpine is 7.4.9-alpine.
+          image: redis:7-alpine@sha256:6ab0b6e7381779332f97b8ca76193e45b0756f38d4c0dcda72dbb3c32061ab99
+          imagePullPolicy: IfNotPresent  # audit CODE-L4 — explicit
           command:
             - sh
             - -c

deploy-k3s/manifests/web/deployment.yaml

@@ -23,8 +23,11 @@ spec:
         app.kubernetes.io/part-of: honeydue
     spec:
       serviceAccountName: web
+      # Explicit pod-level opt-out (audit F11) — defense-in-depth on top of
+      # the ServiceAccount-level setting in rbac.yaml.
+      automountServiceAccountToken: false
       imagePullSecrets:
-        - name: ghcr-credentials
+        - name: gitea-credentials
       securityContext:
         runAsNonRoot: true
         runAsUser: 1001
@@ -43,6 +46,7 @@ spec:
       containers:
         - name: web
           image: IMAGE_PLACEHOLDER  # Replaced by 03-deploy.sh or manual sed
+          imagePullPolicy: IfNotPresent  # audit CODE-L4 — explicit; images are SHA/digest-pinned
           ports:
             - containerPort: 3000
               protocol: TCP

deploy-k3s/manifests/worker/deployment.yaml

@@ -27,8 +27,11 @@ spec:
         app.kubernetes.io/part-of: honeydue
     spec:
       serviceAccountName: worker
+      # Explicit pod-level opt-out (audit F11) — defense-in-depth on top of
+      # the ServiceAccount-level setting in rbac.yaml.
+      automountServiceAccountToken: false
       imagePullSecrets:
-        - name: ghcr-credentials
+        - name: gitea-credentials
       securityContext:
         runAsNonRoot: true
         runAsUser: 1000
@@ -39,6 +42,7 @@ spec:
       containers:
         - name: worker
           image: IMAGE_PLACEHOLDER  # Replaced by 03-deploy.sh
+          imagePullPolicy: IfNotPresent  # audit CODE-L4 — explicit; images are SHA/digest-pinned
           securityContext:
             allowPrivilegeEscalation: false
             readOnlyRootFilesystem: true
@@ -47,64 +51,16 @@ spec:
           envFrom:
             - configMapRef:
                 name: honeydue-config
-          env:
-            - name: POSTGRES_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: POSTGRES_PASSWORD
-            - name: SECRET_KEY
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: SECRET_KEY
-            - name: EMAIL_HOST_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: EMAIL_HOST_PASSWORD
-            - name: FCM_SERVER_KEY
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: FCM_SERVER_KEY
-            - name: REDIS_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: REDIS_PASSWORD
-                  optional: true
-            # B2 (Backblaze) credentials. The worker needs these to delete
-            # B2 objects when the pending_uploads cleanup cron reaps
-            # expired upload sessions. Without them the worker falls back
-            # to local-disk storage which fails on this pod's read-only
-            # root filesystem and disables the cleanup cron.
-            - name: B2_KEY_ID
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: B2_KEY_ID
-            - name: B2_APP_KEY
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: B2_APP_KEY
-            # Observability — workers emit traces (e.g., asynq job spans) to
-            # obs.88oakapps.com over OTLP/HTTP. service.name=honeydue-worker
-            # so api and worker show up as separate services in Jaeger.
-            - name: OBS_TRACES_URL
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: OBS_TRACES_URL
-                  optional: true
-            - name: OBS_INGEST_TOKEN
-              valueFrom:
-                secretKeyRef:
-                  name: honeydue-secrets
-                  key: OBS_INGEST_TOKEN
-                  optional: true
+          # Audit CODE-F8: secrets are NOT injected as environment variables.
+          # Env vars are readable for the life of the pod via /proc/<pid>/environ
+          # and leak into crash dumps / child processes. honeydue-secrets is
+          # mounted read-only at /etc/honeydue/secrets (mode 0400) and the Go
+          # config layer (config.loadFileSecrets) reads each key from its file.
+          # Non-secret config still arrives via the configMapRef above.
           volumeMounts:
+            - name: app-secrets
+              mountPath: /etc/honeydue/secrets
+              readOnly: true
             - name: apns-key
               mountPath: /secrets/apns
               readOnly: true
@@ -124,6 +80,12 @@ spec:
             periodSeconds: 30
             timeoutSeconds: 5
       volumes:
+        # Audit CODE-F8: the whole honeydue-secrets Secret, projected as files.
+        # defaultMode 0400 → readable only by the container's runAsUser (1000).
+        - name: app-secrets
+          secret:
+            secretName: honeydue-secrets
+            defaultMode: 0400
         - name: apns-key
           secret:
             secretName: honeydue-apns-key