honeyDueAPI/docs/deployment/10-secrets-config.md

10 — Secrets & Config

Summary

Non-sensitive config (hostnames, ports, feature flags, etc.) lives in honeydue-config ConfigMap. Sensitive values (DB password, signing keys, API keys) live in honeydue-secrets and honeydue-apns-key Secrets. Container registry auth lives in gitea-credentials (type kubernetes.io/dockerconfigjson). This chapter maps every env var to its source and explains what's stored where.

Structure

flowchart LR
    subgraph SourceWorkstation[Operator workstation]
        ProdEnv[deploy/prod.env]
        Secrets[deploy/secrets/*.txt]
        Registry[deploy/registry.env]
    end

    subgraph K8s[Kubernetes cluster]
        CM[honeydue-config<br/>ConfigMap]
        S1[honeydue-secrets<br/>Secret]
        S2[honeydue-apns-key<br/>Secret]
        S3[gitea-credentials<br/>Secret]
    end

    subgraph Pods
        Api[api pod]
        Admin[admin pod]
        Worker[worker pod]
    end

    ProdEnv -. kubectl create configmap<br/>--from-env-file .-> CM
    Secrets -. kubectl create secret<br/>--from-file/--from-literal .-> S1
    Secrets -. --from-file .-> S2
    Registry -. kubectl create secret docker-registry .-> S3

    CM -- envFrom --> Api & Admin & Worker
    S1 -- env: secretKeyRef --> Api & Worker
    S2 -- volumeMounts --> Api & Worker
    S3 -- imagePullSecrets --> Api & Admin & Worker

ConfigMap: honeydue-config

Built from deploy/prod.env (minus sensitive keys). Contents (58 keys, abbreviated):

ADMIN_PANEL_URL=https://admin.myhoneydue.com
ALLOWED_HOSTS=api.myhoneydue.com,myhoneydue.com
APNS_AUTH_KEY_ID=DISABLED01
APNS_AUTH_KEY_PATH=/secrets/apns/apns_auth_key.p8
APNS_PRODUCTION=false
APNS_TEAM_ID=DISABLED01
APNS_TOPIC=com.myhoneydue.honeyDue
APNS_USE_SANDBOX=false
BASE_URL=https://myhoneydue.com
B2_BUCKET_NAME=honeyDueProd
B2_ENDPOINT=s3.us-east-005.backblazeb2.com
B2_REGION=us-east-005
B2_USE_SSL=true
CORS_ALLOWED_ORIGINS=https://myhoneydue.com,https://admin.myhoneydue.com
DAILY_DIGEST_HOUR=3
DB_HOST=ep-floral-truth-amttbc5a.c-5.us-east-1.aws.neon.tech
DB_MAX_IDLE_CONNS=10
DB_MAX_LIFETIME=600s
DB_MAX_OPEN_CONNS=25
DB_PORT=5432
DB_SSLMODE=require
DEBUG=false
DEFAULT_FROM_EMAIL=noreply@myhoneydue.com
EMAIL_HOST=smtp.fastmail.com
EMAIL_HOST_USER=treytartt@fastmail.com
EMAIL_PORT=587
EMAIL_USE_TLS=true
FEATURE_EMAIL_ENABLED=true
FEATURE_ONBOARDING_EMAILS_ENABLED=true
FEATURE_PDF_REPORTS_ENABLED=true
FEATURE_PUSH_ENABLED=false
FEATURE_WEBHOOKS_ENABLED=true
FEATURE_WORKER_ENABLED=true
NEXT_PUBLIC_API_URL=https://api.myhoneydue.com
OVERDUE_REMINDER_HOUR=15
PORT=8000
POSTGRES_DB=honeyDue
POSTGRES_USER=neondb_owner
REDIS_DB=0
REDIS_URL=redis://redis:6379/0
STATIC_DIR=/app/static
STORAGE_ALLOWED_TYPES=image/jpeg,image/png,image/gif,image/webp,application/pdf
STORAGE_BASE_URL=/uploads
STORAGE_MAX_FILE_SIZE=10485760
STORAGE_UPLOAD_DIR=/app/uploads
TASK_REMINDER_HOUR=14
TIMEZONE=UTC

Plus empty-but-declared keys for optional integrations (Apple/Google auth + IAP).

How pods use it

envFrom:
  - configMapRef:
      name: honeydue-config

Every key in the ConfigMap becomes an env var in the container. envFrom is bulk — no need to enumerate each one.

Changing config

Edit deploy/prod.env locally, regenerate the ConfigMap:

# Simplified; see scripts for the full version
kubectl create configmap honeydue-config -n honeydue \
  --from-env-file=deploy/prod.env \
  --dry-run=client -o yaml | kubectl apply -f -

# Pods don't auto-reload env vars. Restart to pick up changes:
kubectl rollout restart -n honeydue deploy/api deploy/admin deploy/worker

Secret: honeydue-secrets (Opaque)

9 keys:

Key	Purpose
POSTGRES_PASSWORD	Neon DB password
SECRET_KEY	Django-compat signing key (64 chars, base64)
EMAIL_HOST_PASSWORD	Fastmail app password
FCM_SERVER_KEY	FCM push key (currently placeholder, push disabled)
REDIS_PASSWORD	Empty (no auth on in-cluster Redis)
B2_KEY_ID	Backblaze B2 app key ID
B2_APP_KEY	Backblaze B2 app key secret
ADMIN_EMAIL	Next.js admin panel initial admin email
ADMIN_PASSWORD	Next.js admin panel initial admin password

How pods use it

Individual env: entries wire specific Secret keys to env vars:

env:
  - name: POSTGRES_PASSWORD
    valueFrom:
      secretKeyRef:
        name: honeydue-secrets
        key: POSTGRES_PASSWORD
  - name: SECRET_KEY
    valueFrom:
      secretKeyRef:
        name: honeydue-secrets
        key: SECRET_KEY
  # ... etc

This pattern (vs. envFrom: secretRef:) is more explicit — you know exactly which secret keys a pod uses by reading the manifest.

ADMIN_PASSWORD — one-time use

The Go app's internal/database/database.go:519-538 reads ADMIN_EMAIL + ADMIN_PASSWORD at startup. If the admin_users table doesn't have a row for that email, it inserts one with a bcrypt hash of the password. Already-existing rows are not updated.

So:

• First deploy: admin user created
• Subsequent deploys: no-op
• If you want to rotate the initial admin password: do it in the admin panel UI, not by changing ADMIN_PASSWORD

After first deploy you can technically blank ADMIN_PASSWORD in the Secret. Leaving it set is harmless but slightly messy.

Secret: honeydue-apns-key (Opaque)

One file: apns_auth_key.p8. Mounted as a volume into api and worker pods at /secrets/apns/apns_auth_key.p8 (read-only).

Push is currently disabled (FEATURE_PUSH_ENABLED=false), so this .p8 is a throwaway EC P-256 private key generated by openssl genpkey. It passes the Go app's "does this file contain BEGIN PRIVATE KEY" validation but cannot authenticate against Apple.

When push is enabled:

1. Generate a real APNs auth key in Apple Developer console
2. Replace deploy/secrets/apns_auth_key.p8
3. Update APNS_AUTH_KEY_ID, APNS_TEAM_ID, APNS_TOPIC in ConfigMap
4. kubectl create secret generic honeydue-apns-key ... --dry-run=client -o yaml | kubectl apply -f -
5. Set FEATURE_PUSH_ENABLED=true
6. kubectl rollout restart api and worker

Secret: gitea-credentials (docker-registry)

Type kubernetes.io/dockerconfigjson. Contains a base64-encoded Docker config for Gitea registry auth.

Created via:

kubectl create secret docker-registry gitea-credentials \
  --namespace=honeydue \
  --docker-server=gitea.treytartt.com \
  --docker-username=admin \
  --docker-password=<gitea PAT> \
  --dry-run=client -o yaml | kubectl apply -f -

Referenced in every deployment that pulls from Gitea:

spec:
  imagePullSecrets:
    - name: gitea-credentials

When a pod needs to pull an image, the kubelet reads this secret and uses it for the registry authentication.

Source files — what's canonical

The Swarm-era files are still the source of truth for secrets:

File	Contents	Canonical?
deploy/prod.env	All non-sensitive config	Yes
deploy/secrets/postgres_password.txt	Neon DB password	Yes
deploy/secrets/secret_key.txt	App signing key	Yes
deploy/secrets/email_host_password.txt	Fastmail password	Yes
deploy/secrets/fcm_server_key.txt	FCM key (placeholder)	Yes
deploy/secrets/apns_auth_key.p8	APNs key (placeholder)	Yes
deploy/registry.env	Gitea registry auth	Yes
deploy-k3s/manifests/secrets.yaml.example	Template only (never committed with real values)	No — template
In-cluster Secrets	Live state	Derived

Why canonical lives in deploy/ not deploy-k3s/

Historical. We migrated from Swarm to k3s but kept the source files untouched. Rather than move them now (and break any remaining Swarm-era tooling), we use them from the k3s setup scripts as-is.

Future cleanup: move to deploy-k3s/secrets/ for better provenance.

Recreating the cluster secrets

If the k3s cluster is rebuilt, the Secrets need to be recreated from the local source files. Rough procedure:

export KUBECONFIG=~/.kube/honeydue-k3s.yaml

# Namespace first
kubectl create namespace honeydue

# Docker config secret for Gitea
set -a; source deploy/registry.env; set +a
kubectl create secret docker-registry gitea-credentials \
  -n honeydue \
  --docker-server="$REGISTRY" \
  --docker-username="$REGISTRY_USERNAME" \
  --docker-password="$REGISTRY_TOKEN"

# Main secrets bundle
set -a; source deploy/prod.env; set +a
kubectl create secret generic honeydue-secrets -n honeydue \
  --from-literal=POSTGRES_PASSWORD="$(tr -d '\n' < deploy/secrets/postgres_password.txt)" \
  --from-literal=SECRET_KEY="$(tr -d '\n' < deploy/secrets/secret_key.txt)" \
  --from-literal=EMAIL_HOST_PASSWORD="$(tr -d '\n' < deploy/secrets/email_host_password.txt)" \
  --from-literal=FCM_SERVER_KEY="$(tr -d '\n' < deploy/secrets/fcm_server_key.txt)" \
  --from-literal=REDIS_PASSWORD="" \
  --from-literal=B2_KEY_ID="$B2_KEY_ID" \
  --from-literal=B2_APP_KEY="$B2_APP_KEY" \
  --from-literal=ADMIN_EMAIL="$ADMIN_EMAIL" \
  --from-literal=ADMIN_PASSWORD="$ADMIN_PASSWORD"

# APNS key Secret
kubectl create secret generic honeydue-apns-key -n honeydue \
  --from-file=apns_auth_key.p8=deploy/secrets/apns_auth_key.p8

# ConfigMap from prod.env (minus secret keys)
# See deploy-k3s/scripts/02-setup-secrets.sh for the full version
# Simplified:
declare -a args
secret_keys="POSTGRES_PASSWORD SECRET_KEY EMAIL_HOST_PASSWORD FCM_SERVER_KEY REDIS_PASSWORD B2_KEY_ID B2_APP_KEY ADMIN_EMAIL ADMIN_PASSWORD"
while IFS='=' read -r k v; do
  [[ -z "$k" || "$k" =~ ^# ]] && continue
  for sk in $secret_keys; do [[ "$k" == "$sk" ]] && continue 2; done
  args+=(--from-literal="$k=$v")
done < deploy/prod.env
kubectl create configmap honeydue-config -n honeydue "${args[@]}"

The full version with all edge cases is in deploy-k3s/scripts/02-setup-secrets.sh (which was written for the GHCR-era assumption; adapt for Gitea).

Pitfalls

Trailing newlines in secret files

Secret files created by text editors typically end with a newline. If we pass the content directly, the newline becomes part of the secret — a mismatch to what the app expects.

We strip trailing newlines with tr -d '\n' before creating Secrets. If you forget, your DB password will be silently wrong.

Case sensitivity on POSTGRES_DB

POSTGRES_DB=honeyDue must be exactly honeyDue. honeydue (lowercase) fails with database "honeydue" does not exist. Postgres identifiers are case-sensitive if originally quoted at CREATE time.

Placeholder detection

The Swarm-era deploy script rejected values containing CHANGEME, your-, paste_here, etc. When setting up the k3s cluster we had to strip those from prod.env first. If you ever see a pod error about "invalid host" or "invalid key id", check if a placeholder leaked through.

B2_USE_SSL vs STORAGE_USE_SSL

The config has B2_USE_SSL but the Go code reads STORAGE_USE_SSL. See Chapter 9 §Vestigial variable. Setting B2_USE_SSL=false in the ConfigMap does nothing; SSL stays on.

Operator cheat sheet

# Print a ConfigMap as env-file format
kubectl get cm honeydue-config -n honeydue -o jsonpath='{range .data}{"\n"}{end}'

# Edit a ConfigMap interactively (DOES NOT restart pods)
kubectl edit cm honeydue-config -n honeydue

# After editing a ConfigMap, restart pods to pick up
kubectl rollout restart -n honeydue deploy/api deploy/admin deploy/worker

# View a Secret (prints base64 — decode with base64 -d)
kubectl get secret honeydue-secrets -n honeydue -o yaml

# Reveal a specific secret value (DANGER: plaintext to stdout)
kubectl get secret honeydue-secrets -n honeydue \
  -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d

# Update a single secret key
kubectl patch secret honeydue-secrets -n honeydue \
  --type=merge -p "{\"data\":{\"SECRET_KEY\":\"$(echo -n 'newvalue' | base64)\"}}"

References

• Kubernetes ConfigMaps
• Kubernetes Secrets
• Secret types