Trey t
6f303dbbaa
Infrastructure:
- Stack now runs on K3s v1.34.6 HA (3 Hetzner CX33 nodes as managers)
- Traefik DaemonSet + hostNetwork replaces Caddy + ingress mesh
- All manifests in deploy-k3s/manifests/; Swarm config (deploy/) kept
temporarily for reference
Bug fixes surfaced during migration:
- Dockerfile: golang:1.24-alpine -> 1.25-alpine (go.mod requires 1.25)
- cache_service.go: remove sync.Once reassignment from inside Do()
callback (was causing 'unlock of unlocked mutex' fatal after
Redis Ping failure)
- router.go: relax CSP from 'default-src none' to 'default-src self'
+ allowlist fonts.googleapis.com so the marketing landing page CSS
actually loads in browsers
- deploy/scripts/deploy_prod.sh: use docker buildx with
--platform linux/amd64 so arm64 (Apple Silicon) dev machines produce
images runnable on x86_64 Hetzner nodes; fix array expansion under
set -u
- deploy/swarm-stack.prod.yml: fix secret source references to use
top-level aliases (the '\${X_SECRET}' form never actually resolved);
dozzle ports: long-form host_ip is rejected by Swarm, switched to
short-form (bound to 0.0.0.0 with UFW-based loopback restriction);
worker replicas 2 -> 1 (Asynq scheduler singleton)
- deploy-k3s/manifests/admin/deployment.yaml: probe path '/admin/' -> '/'
(Next.js serves at root; /admin/ returned 404 and killed pods);
startupProbe failureThreshold 12 -> 24
- deploy-k3s/manifests/pod-disruption-budgets.yaml: worker minAvailable
1 -> 0 (singleton)
- deploy-k3s/manifests/api/deployment.yaml: startupProbe failureThreshold
12 -> 48 (MigrateWithLock serializes across 3 replicas on first-boot;
real startup takes up to 240s)
- .gitignore: tighten 'api' -> '/api' (was matching deploy-k3s/manifests/api/
and admin/src/app/api/*, hiding legitimate files)
New files:
- deploy-k3s/manifests/traefik-helmchartconfig.yaml: DaemonSet +
hostNetwork override for k3s-bundled Traefik
- deploy-k3s/manifests/ingress/ingress-simple.yaml: plain Ingress
without TLS (CF Flexible SSL) and without middleware
- deploy-k3s/MIGRATION_NOTES.md: operator-facing migration log
Documentation:
- docs/deployment/ — full deployment book, 26 files, ~42k words:
- Part I Overview, infrastructure, orchestrator choice (Ch 0-2)
- Part II Networking, firewall, Cloudflare (Ch 3-4, 13)
- Part III Security, Traefik ingress (Ch 5-6)
- Part IV Services, DB, storage, secrets, registry (Ch 7-11)
- Part V Data flow, deploy process, observability, failures, runbook
(Ch 12, 14-17)
- Part VI Cost, Swarm postmortem, roadmap (Ch 18-20)
- Appendices: glossary, kubectl cheat sheet, file locations,
consolidated citations
- README.md: Production Deployment section replaced with pointer to
the book; Go version bumped to 1.25
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
306 lines
12 KiB
Markdown
306 lines
12 KiB
Markdown
# honeyDue API
|
|
|
|
Go REST API for the honeyDue property management platform. Powers iOS and Android mobile apps built with Kotlin Multiplatform.
|
|
|
|
## Tech Stack
|
|
|
|
- **Language**: Go 1.25
|
|
- **HTTP Framework**: [Echo v4](https://github.com/labstack/echo)
|
|
- **ORM**: [GORM](https://gorm.io/) with PostgreSQL
|
|
- **Background Jobs**: [Asynq](https://github.com/hibiken/asynq) (Redis-backed)
|
|
- **Push Notifications**: APNs ([apns2](https://github.com/sideshow/apns2)) + FCM HTTP API
|
|
- **Caching**: Redis
|
|
- **Logging**: [zerolog](https://github.com/rs/zerolog)
|
|
- **Configuration**: [Viper](https://github.com/spf13/viper)
|
|
- **Admin Panel**: Next.js (separate build target)
|
|
|
|
## Prerequisites
|
|
|
|
- **Go 1.25+** — [install](https://go.dev/dl/)
|
|
- **PostgreSQL 16+** — via Docker (recommended) or [native install](https://www.postgresql.org/download/)
|
|
- **Redis 7+** — via Docker (recommended) or [native install](https://redis.io/docs/getting-started/)
|
|
- **Docker & Docker Compose** — [install](https://docs.docker.com/get-docker/) (recommended for local development)
|
|
- **Make** — pre-installed on macOS; `apt install make` on Linux
|
|
|
|
## Getting Started on a New Machine
|
|
|
|
### Option A: Docker (Recommended)
|
|
|
|
This starts PostgreSQL, Redis, the API server, background worker, and admin panel in containers using the self-contained dev compose file.
|
|
|
|
```bash
|
|
# 1. Clone the repo
|
|
git clone <repo-url>
|
|
cd honeyDueAPI-go
|
|
|
|
# 2. Create your environment file
|
|
cp .env.example .env
|
|
# Edit .env — at minimum set SECRET_KEY to a random 32+ char string
|
|
|
|
# 3. Build and start all services
|
|
make docker-dev
|
|
|
|
# 4. Seed the database with lookup data (required)
|
|
docker exec -i honeydue-db psql -U honeydue -d honeydue < seeds/001_lookups.sql
|
|
|
|
# 5. (Optional) Seed test data — creates test users, residences, tasks
|
|
docker exec -i honeydue-db psql -U honeydue -d honeydue < seeds/002_test_data.sql
|
|
docker exec -i honeydue-db psql -U honeydue -d honeydue < seeds/003_task_templates.sql
|
|
|
|
# 6. Verify the API is running
|
|
curl http://localhost:8000/api/health/
|
|
```
|
|
|
|
The API is now available at `http://localhost:8000`.
|
|
|
|
### Option B: Run Locally (No Docker for the API)
|
|
|
|
Use Docker for PostgreSQL and Redis, but run the Go server natively for faster iteration.
|
|
|
|
```bash
|
|
# 1. Clone and enter the repo
|
|
git clone <repo-url>
|
|
cd honeyDueAPI-go
|
|
|
|
# 2. Install Go dependencies
|
|
make deps
|
|
|
|
# 3. Start PostgreSQL and Redis via Docker
|
|
docker compose -f docker-compose.dev.yml up -d db redis
|
|
|
|
# 4. Create your environment file
|
|
cp .env.example .env
|
|
# Edit .env:
|
|
# - Set SECRET_KEY to a random 32+ char string
|
|
# - Set DB_HOST=localhost
|
|
# - Set DB_PORT=5433 (docker-compose maps 5433 externally)
|
|
# - Set REDIS_URL=redis://localhost:6379/0
|
|
|
|
# 5. Seed the database
|
|
psql -h localhost -p 5433 -U honeydue -d honeydue < seeds/001_lookups.sql
|
|
psql -h localhost -p 5433 -U honeydue -d honeydue < seeds/002_test_data.sql
|
|
psql -h localhost -p 5433 -U honeydue -d honeydue < seeds/003_task_templates.sql
|
|
|
|
# 6. Run the API server
|
|
make run
|
|
|
|
# 7. (Optional) Run the background worker in a separate terminal
|
|
make run-worker
|
|
```
|
|
|
|
### Option C: Fully Native (No Docker)
|
|
|
|
Install PostgreSQL and Redis natively on your machine.
|
|
|
|
```bash
|
|
# macOS with Homebrew
|
|
brew install postgresql@16 redis
|
|
|
|
# Start services
|
|
brew services start postgresql@16
|
|
brew services start redis
|
|
|
|
# Create the database
|
|
createdb honeydue
|
|
|
|
# Then follow Option B steps 2-7, using:
|
|
# DB_HOST=localhost, DB_PORT=5432, POSTGRES_USER=<your-user>, POSTGRES_PASSWORD=<your-password>
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
Copy `.env.example` to `.env` and configure:
|
|
|
|
| Variable | Description | Default | Required |
|
|
|----------|-------------|---------|----------|
|
|
| `PORT` | Server port | `8000` | No |
|
|
| `DEBUG` | Enable debug logging | `true` | No |
|
|
| `SECRET_KEY` | JWT signing secret (32+ chars) | — | **Yes** |
|
|
| `POSTGRES_DB` | Database name | `honeydue` | Yes |
|
|
| `POSTGRES_USER` | Database user | `postgres` | Yes |
|
|
| `POSTGRES_PASSWORD` | Database password | — | Yes |
|
|
| `DB_HOST` | Database host | `localhost` | Yes |
|
|
| `DB_PORT` | Database port | `5432` | Yes |
|
|
| `REDIS_URL` | Redis connection URL | `redis://localhost:6379/0` | Yes |
|
|
| `EMAIL_HOST` | SMTP server | `smtp.gmail.com` | For email |
|
|
| `EMAIL_HOST_USER` | SMTP username | — | For email |
|
|
| `EMAIL_HOST_PASSWORD` | SMTP app password | — | For email |
|
|
| `APNS_AUTH_KEY_PATH` | Path to APNs `.p8` key file | — | For iOS push |
|
|
| `APNS_AUTH_KEY_ID` | APNs key ID | — | For iOS push |
|
|
| `APNS_TEAM_ID` | Apple Team ID | — | For iOS push |
|
|
| `FCM_SERVER_KEY` | Firebase server key | — | For Android push |
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
honeyDueAPI-go/
|
|
├── cmd/
|
|
│ ├── api/main.go # API server entry point
|
|
│ └── worker/main.go # Background worker entry point
|
|
├── internal/
|
|
│ ├── config/ # Viper configuration
|
|
│ ├── database/ # PostgreSQL connection setup
|
|
│ ├── models/ # GORM models (map to PostgreSQL tables)
|
|
│ ├── repositories/ # Data access layer
|
|
│ ├── services/ # Business logic
|
|
│ ├── handlers/ # HTTP handlers (Echo)
|
|
│ ├── middleware/ # Auth, timezone, logging middleware
|
|
│ ├── router/ # Route registration
|
|
│ ├── dto/ # Request/Response DTOs
|
|
│ │ ├── requests/ # Incoming request structs
|
|
│ │ └── responses/ # Outgoing response structs
|
|
│ ├── task/ # Centralized task logic
|
|
│ │ ├── predicates/ # IsCompleted, IsOverdue, etc.
|
|
│ │ ├── scopes/ # GORM query scopes
|
|
│ │ └── categorization/ # Kanban column assignment
|
|
│ ├── apperrors/ # Structured error types
|
|
│ ├── push/ # APNs + FCM push notification clients
|
|
│ ├── worker/ # Asynq background jobs
|
|
│ ├── i18n/ # Internationalization (en, es, fr)
|
|
│ ├── validator/ # Input validation
|
|
│ ├── monitoring/ # Health checks
|
|
│ └── integration/ # Integration + contract tests
|
|
├── admin/ # Next.js admin panel
|
|
├── migrations/ # SQL migration files
|
|
├── seeds/ # Seed data (lookups, test users)
|
|
├── templates/emails/ # Email templates
|
|
├── docs/ # API docs, OpenAPI spec
|
|
├── docker-compose.yml # Production / Docker Swarm config
|
|
├── docker-compose.dev.yml # Self-contained local dev config
|
|
├── Dockerfile # Multi-stage build (api, worker, admin)
|
|
├── Makefile # Build, test, Docker commands
|
|
└── .env.example # Environment variable template
|
|
```
|
|
|
|
## Development
|
|
|
|
### Common Commands
|
|
|
|
```bash
|
|
make run # Run API server
|
|
make run-worker # Run background worker
|
|
make deps # Install/tidy Go dependencies
|
|
make build # Build API binary
|
|
make build-all # Build API + worker binaries
|
|
make fmt # Format code
|
|
make vet # Vet code
|
|
make lint # Run golangci-lint
|
|
make clean # Remove build artifacts
|
|
```
|
|
|
|
### Testing
|
|
|
|
```bash
|
|
make test # Run all tests with race detection + coverage
|
|
make test-coverage # Run tests and generate HTML coverage report
|
|
make contract-test # Run route + KMP contract validation tests
|
|
|
|
# Run specific packages
|
|
go test ./internal/handlers -v
|
|
go test -run TestTaskHandler_CreateTask ./internal/handlers
|
|
```
|
|
|
|
### Docker
|
|
|
|
Two compose files:
|
|
- **`docker-compose.dev.yml`** — self-contained local dev (build from source, `container_name`, `depends_on`, dev defaults)
|
|
- **`docker-compose.yml`** — production / Docker Swarm (image refs, `deploy:` sections, overlay network)
|
|
|
|
```bash
|
|
# Dev
|
|
make docker-dev # Build and start all dev containers (foreground)
|
|
make docker-up # Start dev containers (detached)
|
|
make docker-down # Stop dev containers
|
|
make docker-logs # Tail dev container logs
|
|
make docker-restart # Restart dev containers
|
|
|
|
# Prod
|
|
make docker-build-prod # Build production images (api, worker, admin)
|
|
```
|
|
|
|
### Database Migrations
|
|
|
|
```bash
|
|
make migrate-up # Apply pending migrations
|
|
make migrate-down # Roll back last migration
|
|
make migrate-create name=add_column # Create a new migration pair
|
|
```
|
|
|
|
## Seed Data
|
|
|
|
Seed files in `seeds/`:
|
|
|
|
| File | Purpose | Required |
|
|
|------|---------|----------|
|
|
| `001_lookups.sql` | Residence types, task categories, priorities, frequencies, contractor specialties | **Yes** |
|
|
| `002_test_data.sql` | Test users, residences, tasks, contractors, documents, notifications | Dev only |
|
|
| `003_task_templates.sql` | Pre-built task templates for onboarding | Dev only |
|
|
| `003_admin_user.sql` | Admin panel user | If using admin |
|
|
|
|
### Test Users
|
|
|
|
All test users have password: `password123`
|
|
|
|
| Username | Email | Tier | Notes |
|
|
|----------|-------|------|-------|
|
|
| admin | admin@example.com | Pro | Admin user |
|
|
| john | john@example.com | Pro | Owns 2 residences |
|
|
| jane | jane@example.com | Free | Owns 1 residence, shared access |
|
|
| bob | bob@example.com | Free | Owns 1 residence |
|
|
|
|
## API Documentation
|
|
|
|
- **OpenAPI spec**: `docs/openapi.yaml` (81 paths, 104 operations, 81 schemas)
|
|
- **Health check**: `GET /api/health/`
|
|
- **Auth**: `POST /api/auth/login/`, `POST /api/auth/register/`
|
|
- **Static data**: `GET /api/static_data/` (ETag-cached lookups)
|
|
|
|
All protected endpoints require an `Authorization: Token <token>` header.
|
|
|
|
## Production Deployment
|
|
|
|
Production runs on a **3-node K3s HA cluster** on Hetzner Cloud, fronted
|
|
by Cloudflare, with Neon Postgres, Backblaze B2, and a self-hosted Gitea
|
|
container registry. See the full deployment book for every detail:
|
|
|
|
**→ [docs/deployment/](./docs/deployment/README.md) — The Deployment Book**
|
|
|
|
26 chapters and ~42,000 words covering:
|
|
|
|
- **Part I — The System**: overview, Hetzner infrastructure, why K3s
|
|
(and not Swarm, full Kubernetes, or Nomad)
|
|
- **Part II — Networking**: Flannel VXLAN, CoreDNS, kube-proxy, every
|
|
UFW rule on every node, Cloudflare DNS setup
|
|
- **Part III — Security**: RBAC, Pod Security, secrets, TLS chain
|
|
- **Part IV — Workloads**: api, admin, worker, redis per-service deep
|
|
dives; Neon Postgres config; Backblaze B2 storage; Gitea registry
|
|
- **Part V — Operation**: end-to-end data flow, deploy process,
|
|
observability, failure modes, operator runbook
|
|
- **Part VI — Context**: cost breakdown, postmortem of the bugs from
|
|
the Swarm→K3s migration, roadmap
|
|
|
|
Quick links:
|
|
|
|
- **Runbook** — [docs/deployment/17-runbook.md](./docs/deployment/17-runbook.md) — 22 common ops procedures
|
|
- **kubectl cheat sheet** — [docs/deployment/appendices/b-commands.md](./docs/deployment/appendices/b-commands.md)
|
|
- **Deploy process** — [docs/deployment/14-deployment-process.md](./docs/deployment/14-deployment-process.md) — build → push → rollout
|
|
- **Failure modes** — [docs/deployment/16-failure-modes.md](./docs/deployment/16-failure-modes.md) — what happens when X dies
|
|
- **Swarm postmortem** — [docs/deployment/19-postmortem-swarm.md](./docs/deployment/19-postmortem-swarm.md) — why we migrated
|
|
|
|
Operational state lives under:
|
|
|
|
- `deploy-k3s/manifests/` — Kubernetes manifests (apply with `kubectl`)
|
|
- `deploy-k3s/MIGRATION_NOTES.md` — notes from the Swarm → K3s migration
|
|
- `deploy/` — legacy Swarm config (retained temporarily; to be removed)
|
|
|
|
## Related Projects
|
|
|
|
- **Deployment Book**: [`docs/deployment/`](./docs/deployment/README.md) — full production operations reference
|
|
- **Mobile App (KMM)**: `../HoneyDueKMM` — Kotlin Multiplatform iOS/Android client
|
|
- **Task Logic Docs**: `docs/TASK_LOGIC_ARCHITECTURE.md` — required reading before task-related work
|
|
- **Push Notification Docs**: `docs/PUSH_NOTIFICATIONS.md`
|
|
|
|
## License
|
|
|
|
Proprietary — honeyDue
|