backend: GDPR export + retention cleanups + worker metrics (BE-1/2/3)
Backend CI / Test (push) Has been cancelled
Backend CI / Contract Tests (push) Has been cancelled
Backend CI / Lint (push) Has been cancelled
Backend CI / Secret Scanning (push) Has been cancelled
Backend CI / Build (push) Has been cancelled

BE-3 observability: expose the worker's Prometheus metrics on :6060/metrics
(apns/fcm/asynq histograms + a new cache_ops_total counter were recorded all
along but never scraped — which is why those dashboard panels read empty); add
the worker containerPort, the vmagent worker scrape job, and two additive
NetworkPolicies. Instrument cache Get/Set hit/miss.

BE-2 retention: three periodic Asynq cleanup crons mirroring the reminder-log
cleanup — notifications (90d), webhook dedup log (180d), audit_log (365d).

BE-1 GDPR data export: POST /api/auth/export/ enqueues a low-priority Asynq job
that gathers all of the user's data (owned residences + their tasks/contractors/
documents/share-codes, plus profile/notifications/prefs/push-tokens/subscription/
audit log), zips one JSON file per category, and emails it as an attachment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Trey T
2026-06-08 22:15:26 -05:00
parent 3b2ea9959a
commit b54493f785
14 changed files with 421 additions and 13 deletions
+21
View File
@@ -1,6 +1,7 @@
package prom
import (
"net/http"
"strconv"
"time"
@@ -54,6 +55,11 @@ var (
Help: "Duration of asynq background job execution in seconds.",
Buckets: []float64{0.01, 0.05, 0.1, 0.5, 1, 5, 10, 30, 60, 300},
}, []string{"task_type", "result"})
cacheOps = prometheus.NewCounterVec(prometheus.CounterOpts{
Name: "cache_ops_total",
Help: "Redis cache operations by type and result.",
}, []string{"operation", "result"}) // operation: get|set; result: hit|miss|ok|error
)
func init() {
@@ -67,6 +73,7 @@ func init() {
apnsSendDuration,
fcmSendDuration,
asynqJobDuration,
cacheOps,
)
}
@@ -77,6 +84,20 @@ func Handler() echo.HandlerFunc {
return echo.WrapHandler(h)
}
// HTTPHandler returns a net/http handler bound to the package Registry, for the
// worker's plain http.ServeMux (the api uses Handler() for Echo). This is what
// lets the worker's apns/fcm/asynq histograms actually get scraped — they were
// recorded all along but the worker exposed no /metrics endpoint.
func HTTPHandler() http.Handler {
return promhttp.HandlerFor(Registry, promhttp.HandlerOpts{Registry: Registry})
}
// ObserveCacheOp records a Redis cache operation. operation is "get" or "set";
// result is "hit"/"miss"/"error" for gets and "ok"/"error" for sets.
func ObserveCacheOp(operation, result string) {
cacheOps.WithLabelValues(operation, result).Inc()
}
// HTTPMiddleware records http_request_duration_seconds for every request,
// labeled by Echo route pattern, method, and status code.
func HTTPMiddleware() echo.MiddlewareFunc {