perf(task): offload completion notification fan-out to Asynq worker
POST /api/task-completions/ was spending ~1.5-1.75s synchronously on
APNs push + SMTP email + B2 image fetches inside sendTaskCompletedNotification.
Per-user loop made it scale linearly with residence membership; one image
attached + one residence user is the 1.75s baseline observed in the live
honeydue-eli5-overview Grafana panel.
Replace the inline call (and the fire-and-forget goroutine in QuickComplete,
which violated the project's "no goroutines in handlers" rule) with an
Asynq job:
- new task type notification:task_completed (worker/scheduler.go)
- new payload {task_id, completion_id} — IDs only, worker re-reads
canonical state from Postgres so concurrent edits between enqueue
and dequeue are reflected
- new HandleTaskCompletedNotification on jobs.Handler delegates to
TaskService.SendTaskCompletedNotificationByID
- new dispatchTaskCompletedNotification in task_service.go picks
between enqueue (preferred) and inline (fallback) when Redis is
unreachable or the enqueuer isn't wired (tests / local dev)
Other changes required to wire it up:
- widen worker.NewTaskClient signature to accept asynq.RedisClientOpt
so the file-mounted Redis password (audit HIGH-1) can be supplied;
no prior callers, no breakage
- extend worker.Enqueuer interface with EnqueueTaskCompletedNotification
- add TaskEnqueuer field to router.Dependencies; wire from cmd/api/main.go
with the standard typed-nil interface guard
- wire a worker-side TaskService in cmd/worker/main.go so the handler
can use the shared SendTaskCompletedNotificationByID implementation
(storage service shared with the existing upload-cleanup wiring)
Expected impact on POST /api/task-completions/ p50:
~1.75s -> ~120-170ms (DB + tx + Asynq enqueue only)
Notifications still deliver; they just go via the worker instead of in
the request path. MaxRetry=3; "row not found" returns nil so a deleted
task/completion doesn't churn the retry loop.
All 31 test packages pass. No DB migrations.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -9,6 +9,7 @@ import (
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/hibiken/asynq"
|
||||
"github.com/rs/zerolog/log"
|
||||
"gorm.io/gorm"
|
||||
|
||||
@@ -20,6 +21,7 @@ import (
|
||||
"github.com/treytartt/honeydue-api/internal/router"
|
||||
"github.com/treytartt/honeydue-api/internal/services"
|
||||
"github.com/treytartt/honeydue-api/internal/tracing"
|
||||
"github.com/treytartt/honeydue-api/internal/worker"
|
||||
"github.com/treytartt/honeydue-api/pkg/utils"
|
||||
)
|
||||
|
||||
@@ -194,6 +196,28 @@ func main() {
|
||||
Msg("Push notification client initialized")
|
||||
}
|
||||
|
||||
// Initialize Asynq enqueuer (api-side). Used by services that move
|
||||
// long-running work off the request path (currently: task-completion
|
||||
// notification fan-out). Same Redis as cmd/worker — file-mounted password
|
||||
// applied separately because cfg.Redis.URL does not embed it (audit HIGH-1).
|
||||
var taskEnqueuer *worker.TaskClient
|
||||
if redisOpt, parseErr := asynq.ParseRedisURI(cfg.Redis.URL); parseErr != nil {
|
||||
log.Warn().Err(parseErr).Msg("Failed to parse Redis URL for Asynq enqueuer — completion notifications will run inline")
|
||||
} else if clientOpt, ok := redisOpt.(asynq.RedisClientOpt); ok {
|
||||
if cfg.Redis.Password != "" {
|
||||
clientOpt.Password = cfg.Redis.Password
|
||||
}
|
||||
taskEnqueuer = worker.NewTaskClient(clientOpt)
|
||||
defer func() {
|
||||
if cerr := taskEnqueuer.Close(); cerr != nil {
|
||||
log.Warn().Err(cerr).Msg("Failed to close Asynq enqueuer on shutdown")
|
||||
}
|
||||
}()
|
||||
log.Info().Msg("Asynq enqueuer initialized")
|
||||
} else {
|
||||
log.Warn().Msg("Redis opt is not RedisClientOpt — Asynq enqueuer skipped; completion notifications will run inline")
|
||||
}
|
||||
|
||||
// Setup router with dependencies (includes admin panel at /admin)
|
||||
deps := &router.Dependencies{
|
||||
DB: db,
|
||||
@@ -205,6 +229,12 @@ func main() {
|
||||
StorageService: storageService,
|
||||
MonitoringService: monitoringService,
|
||||
}
|
||||
// Only assign the enqueuer when we actually constructed one. Assigning a
|
||||
// nil *worker.TaskClient directly would create a typed-nil interface that
|
||||
// fails the `if deps.TaskEnqueuer != nil` check in router.SetupRouter.
|
||||
if taskEnqueuer != nil {
|
||||
deps.TaskEnqueuer = taskEnqueuer
|
||||
}
|
||||
e := router.SetupRouter(deps)
|
||||
|
||||
// Create HTTP server
|
||||
|
||||
Reference in New Issue
Block a user