Cut /api/tasks/ p99 from ~2500ms toward ~150-300ms
Backend CI / Test (push) Has been cancelled
Backend CI / Contract Tests (push) Has been cancelled
Backend CI / Build (push) Has been cancelled
Backend CI / Lint (push) Has been cancelled
Backend CI / Secret Scanning (push) Has been cancelled

Stack of optimizations against the same Hetzner→Neon transatlantic link.
The trace revealed every visible ms was network/proxy overhead — DB
execution itself is sub-millisecond per query (verified via EXPLAIN
ANALYZE: index scans on every hot path).

Connection layer:
- DB_HOST → Neon pooler endpoint (-pooler suffix). PgBouncer
  transaction-mode keeps backend Postgres connections warm so we no
  longer pay the ~110ms Postgres-startup RTT on cold queries.
- GORM pool tuned: MaxIdleConns 10→20, MaxLifetime 600s→1800s,
  MaxIdleTime added (default 0 = never close idle).
- Eager pool warm-up at boot via parallel pings — first user request
  no longer pays the ~440ms TCP+TLS+startup handshake.
- Redis maxmemory-policy noeviction → allkeys-lru. Cache writes will
  evict cold keys instead of erroring at the 256MB limit.

Auth layer:
- TokenCacheTTL 5min → 1 hour (Redis token cache).
- UserCacheTTL 30s → 5min (in-memory User cache, per pod).
- UserCache gains a 5,000-entry LRU cap so a flood of unique users
  can't blow up pod RSS. ~5MB worst-case per pod.
- Token + user lookup collapsed from 2 GORM Preload queries into a
  single INNER JOIN. Saves 1 RTT per cold-cache request.
- Auth middleware's m.db.* now use db.WithContext(ctx) so the SQL
  spans nest under the parent HTTP request in Jaeger.

Service layer:
- TaskService.ListTasks: replaced two-step
  FindResidenceIDsByUser → GetKanbanDataForMultipleResidences
  with a single GetKanbanDataForUser that uses a Postgres subquery
  for residence-access. One round-trip instead of two.
- New CacheService residence-IDs cache: \"residence_ids_user:<id>\"
  with 5-min TTL. Wired into Task/Residence/Contractor/Document
  services for the four hot read paths that need this list.
- Cache invalidation on every relevant mutation: CreateResidence,
  DeleteResidence, JoinWithCode, RemoveUser. DeleteResidence
  invalidates every member of the residence, not just the owner.

What this stacks up to (Hetzner→Neon, before US migration):
  Path                                 Before        After (target)
  Cache-warm authed read               ~800ms        ~100-200ms
  Cache-cold authed read (1st in 1hr)  ~2500ms       ~500-700ms
  First request after deploy           ~2500ms       ~700-900ms

The endgame US-region migration on top of this gets us to ~30-50ms
warm-cache, but we're shippable at ~150ms warm right now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Trey t
2026-04-25 17:13:50 -05:00
parent 9410da7497
commit 88fb1751c7
15 changed files with 443 additions and 59 deletions
+12 -18
View File
@@ -37,6 +37,12 @@ type TaskService struct {
notificationService *NotificationService
emailService *EmailService
storageService *StorageService
cache *CacheService
}
// SetCacheService wires Redis caching for residence-ID lookups.
func (s *TaskService) SetCacheService(cache *CacheService) {
s.cache = cache
}
// NewTaskService creates a new task service
@@ -108,23 +114,11 @@ func (s *TaskService) ListTasks(ctx context.Context, userID uint, daysThreshold
daysThreshold = 30 // Default
}
// Get all residence IDs accessible to user (lightweight - no preloads)
residenceIDs, err := s.residenceRepo.WithContext(ctx).FindResidenceIDsByUser(userID)
if err != nil {
return nil, apperrors.Internal(err)
}
if len(residenceIDs) == 0 {
// Return empty kanban board
return &responses.KanbanBoardResponse{
Columns: []responses.KanbanColumnResponse{},
DaysThreshold: daysThreshold,
ResidenceID: "all",
}, nil
}
// Get kanban data aggregated across all residences using user's timezone-aware time
board, err := s.taskRepo.WithContext(ctx).GetKanbanDataForMultipleResidences(residenceIDs, daysThreshold, now)
// Single-round-trip variant: residence-access subquery is folded into the
// task fetch on the Postgres side instead of a separate Go-side round-trip.
// Saves ~110ms on Hetzner→Neon. Empty result naturally handles the
// "user has no residences" case as an empty board.
board, err := s.taskRepo.WithContext(ctx).GetKanbanDataForUser(userID, daysThreshold, now)
if err != nil {
return nil, apperrors.Internal(err)
}
@@ -1025,7 +1019,7 @@ func (s *TaskService) GetCompletion(ctx context.Context, completionID, userID ui
// ListCompletions lists all task completions for a user
func (s *TaskService) ListCompletions(ctx context.Context, userID uint) ([]responses.TaskCompletionResponse, error) {
// Get all residence IDs (lightweight - no preloads)
residenceIDs, err := s.residenceRepo.WithContext(ctx).FindResidenceIDsByUser(userID)
residenceIDs, err := cachedResidenceIDsForUser(ctx, s.cache, s.residenceRepo, userID)
if err != nil {
return nil, apperrors.Internal(err)
}