perf(task): offload completion notification fan-out to Asynq worker
POST /api/task-completions/ was spending ~1.5-1.75s synchronously on
APNs push + SMTP email + B2 image fetches inside sendTaskCompletedNotification.
Per-user loop made it scale linearly with residence membership; one image
attached + one residence user is the 1.75s baseline observed in the live
honeydue-eli5-overview Grafana panel.
Replace the inline call (and the fire-and-forget goroutine in QuickComplete,
which violated the project's "no goroutines in handlers" rule) with an
Asynq job:
- new task type notification:task_completed (worker/scheduler.go)
- new payload {task_id, completion_id} — IDs only, worker re-reads
canonical state from Postgres so concurrent edits between enqueue
and dequeue are reflected
- new HandleTaskCompletedNotification on jobs.Handler delegates to
TaskService.SendTaskCompletedNotificationByID
- new dispatchTaskCompletedNotification in task_service.go picks
between enqueue (preferred) and inline (fallback) when Redis is
unreachable or the enqueuer isn't wired (tests / local dev)
Other changes required to wire it up:
- widen worker.NewTaskClient signature to accept asynq.RedisClientOpt
so the file-mounted Redis password (audit HIGH-1) can be supplied;
no prior callers, no breakage
- extend worker.Enqueuer interface with EnqueueTaskCompletedNotification
- add TaskEnqueuer field to router.Dependencies; wire from cmd/api/main.go
with the standard typed-nil interface guard
- wire a worker-side TaskService in cmd/worker/main.go so the handler
can use the shared SendTaskCompletedNotificationByID implementation
(storage service shared with the existing upload-cleanup wiring)
Expected impact on POST /api/task-completions/ p50:
~1.75s -> ~120-170ms (DB + tx + Asynq enqueue only)
Notifications still deliver; they just go via the worker instead of in
the request path. MaxRetry=3; "row not found" returns nil so a deleted
task/completion doesn't churn the retry loop.
All 31 test packages pass. No DB migrations.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
+29
-4
@@ -23,6 +23,7 @@ import (
|
||||
"github.com/treytartt/honeydue-api/internal/repositories"
|
||||
"github.com/treytartt/honeydue-api/internal/services"
|
||||
"github.com/treytartt/honeydue-api/internal/tracing"
|
||||
"github.com/treytartt/honeydue-api/internal/worker"
|
||||
"github.com/treytartt/honeydue-api/internal/worker/jobs"
|
||||
"github.com/treytartt/honeydue-api/pkg/utils"
|
||||
)
|
||||
@@ -180,11 +181,15 @@ func main() {
|
||||
// Create job handler
|
||||
jobHandler := jobs.NewHandler(db, pushClient, emailService, notificationService, cfg)
|
||||
|
||||
// Wire upload service for the pending_uploads cleanup cron. Storage may
|
||||
// be local-disk (no S3 backend), in which case the upload service stays
|
||||
// nil and the cleanup handler no-ops. Cache is optional — the cleanup
|
||||
// path doesn't rate-limit and works fine with a nil cache.
|
||||
// Wire upload service for the pending_uploads cleanup cron AND share the
|
||||
// underlying storage service with the TaskService below so the worker can
|
||||
// load completion images for email embedding. Storage may be local-disk
|
||||
// (no S3 backend), in which case the upload service stays nil and the
|
||||
// cleanup handler no-ops. Cache is optional — the cleanup path doesn't
|
||||
// rate-limit and works fine with a nil cache.
|
||||
var sharedStorageService *services.StorageService
|
||||
if storageService, sErr := services.NewStorageService(&cfg.Storage); sErr == nil {
|
||||
sharedStorageService = storageService
|
||||
if s3 := storageService.S3Backend(); s3 != nil {
|
||||
pendingUploadRepo := repositories.NewPendingUploadRepository(db)
|
||||
uploadService := services.NewUploadService(pendingUploadRepo, s3, &cfg.Storage, nil)
|
||||
@@ -194,6 +199,25 @@ func main() {
|
||||
log.Warn().Err(sErr).Msg("Failed to initialize storage service for upload cleanup; cleanup cron will no-op")
|
||||
}
|
||||
|
||||
// Wire a TaskService for the task-completed notification handler. The
|
||||
// worker re-creates this (vs. importing the api's wired instance) because
|
||||
// each binary owns its own dependency graph. The handler is fully nil-safe
|
||||
// — if any of the wired services are absent, the corresponding side of
|
||||
// notification delivery (push or email) is skipped.
|
||||
taskRepo := repositories.NewTaskRepository(db)
|
||||
residenceRepo := repositories.NewResidenceRepository(db)
|
||||
workerTaskService := services.NewTaskService(taskRepo, residenceRepo)
|
||||
if notificationService != nil {
|
||||
workerTaskService.SetNotificationService(notificationService)
|
||||
}
|
||||
if emailService != nil {
|
||||
workerTaskService.SetEmailService(emailService)
|
||||
}
|
||||
if sharedStorageService != nil {
|
||||
workerTaskService.SetStorageService(sharedStorageService)
|
||||
}
|
||||
jobHandler.SetTaskService(workerTaskService)
|
||||
|
||||
// Create Asynq mux and register handlers
|
||||
mux := asynq.NewServeMux()
|
||||
|
||||
@@ -208,6 +232,7 @@ func main() {
|
||||
mux.HandleFunc(jobs.TypeOnboardingEmails, jobHandler.HandleOnboardingEmails)
|
||||
mux.HandleFunc(jobs.TypeReminderLogCleanup, jobHandler.HandleReminderLogCleanup)
|
||||
mux.HandleFunc(jobs.TypeUploadCleanup, jobHandler.HandleUploadCleanup)
|
||||
mux.HandleFunc(worker.TypeTaskCompletedNotification, jobHandler.HandleTaskCompletedNotification)
|
||||
|
||||
// Register email job handlers (welcome, verification, password reset, password changed)
|
||||
if emailService != nil {
|
||||
|
||||
Reference in New Issue
Block a user