feat: add Django web app, CloudKit sync, dashboard, and game_datetime_utc export

Adds the full Django application layer on top of sportstime_parser:
- core: Sport, Team, Stadium, Game models with aliases and league structure
- scraper: orchestration engine, adapter, job management, Celery tasks
- cloudkit: CloudKit sync client, sync state tracking, sync jobs
- dashboard: staff dashboard for monitoring scrapers, sync, review queue
- notifications: email reports for scrape/sync results
- Docker setup for deployment (Dockerfile, docker-compose, entrypoint)

Game exports now use game_datetime_utc (ISO 8601 UTC) instead of
venue-local date+time strings, matching the canonical format used
by the iOS app.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Trey t
2026-02-19 14:04:27 -06:00
parent 4353d5943c
commit 63acf7accb
114 changed files with 13070 additions and 887 deletions

131
CLAUDE.md Normal file
View File

@@ -0,0 +1,131 @@
# CLAUDE.md
This file provides context for Claude Code when working on this project.
## Project Overview
SportsTime is a Django-based sports data pipeline that scrapes game schedules from official sources, normalizes the data, stores it in PostgreSQL, and syncs to CloudKit for iOS app consumption.
## Architecture
```
┌─────────────────┐ ┌──────────────┐ ┌─────────────┐ ┌──────────┐
│ Data Sources │ ──▶ │ Scrapers │ ──▶ │ PostgreSQL │ ──▶ │ CloudKit │
│ (ESPN, leagues) │ │ (sportstime_ │ │ (Django) │ │ (iOS) │
└─────────────────┘ │ parser) │ └─────────────┘ └──────────┘
└──────────────┘
```
## Key Directories
- `core/` - Django models: Sport, Team, Stadium, Game, Conference, Division, Aliases
- `scraper/` - Scraper orchestration, adapter, job management
- `sportstime_parser/` - Standalone scraper library (ESPN, league APIs)
- `cloudkit/` - CloudKit sync client and job management
- `dashboard/` - Staff dashboard for monitoring and controls
- `templates/` - Django templates for dashboard UI
## Data Flow
1. **Scraper runs** (manual or scheduled via Celery Beat)
2. **sportstime_parser** fetches from ESPN/league APIs
3. **Adapter** normalizes data and resolves team/stadium names
4. **Django models** store normalized data with CloudKit sync flags
5. **CloudKit sync** pushes pending records to iCloud
## Models Hierarchy
```
Sport
├── Conference
│ └── Division
│ └── Team (has TeamAliases)
├── Stadium (has StadiumAliases)
└── Game (references Team, Stadium)
```
## Name Resolution
Team and stadium names from scraped data are resolved via:
1. Direct ID match (canonical IDs from scraper)
2. Database aliases (TeamAlias/StadiumAlias with date validity)
3. Direct name/abbreviation match
Aliases support validity dates for historical names (e.g., team relocations, stadium naming rights).
## Common Tasks
### Run a scraper
```bash
docker-compose exec web python manage.py shell
>>> from scraper.tasks import run_scraper_task
>>> run_scraper_task.delay(config_id)
```
### Check scraper status
Visit `/dashboard/scraper-status/` or check `ScrapeJob` model.
### Add team/stadium alias
Use Django admin at `/admin/core/teamalias/` or `/admin/core/stadiumalias/`.
### Export/Import data
All admin models support import/export (JSON, CSV, XLSX) via django-import-export.
### Sync to CloudKit
```bash
docker-compose exec web python manage.py shell
>>> from cloudkit.tasks import run_cloudkit_sync
>>> run_cloudkit_sync.delay(config_id)
```
## Environment
- **Docker Compose** for local development
- **PostgreSQL** database
- **Redis** for Celery broker
- **Celery** for async tasks and scheduled jobs
## Key Files
- `sportstime/settings.py` - Django settings
- `scraper/engine/adapter.py` - Bridges sportstime_parser to Django
- `scraper/engine/db_alias_loader.py` - Database alias resolution
- `core/resources.py` - Import/export resource definitions
- `docker-compose.yml` - Container orchestration
## Supported Sports
| Code | Sport | Season Type |
|------|-------|-------------|
| nba | NBA Basketball | split (Oct-Jun) |
| mlb | MLB Baseball | calendar (Mar-Oct) |
| nfl | NFL Football | split (Sep-Feb) |
| nhl | NHL Hockey | split (Oct-Jun) |
| mls | MLS Soccer | calendar (Feb-Nov) |
| wnba | WNBA Basketball | calendar (May-Sep) |
| nwsl | NWSL Soccer | calendar (Mar-Nov) |
## Testing
```bash
docker-compose exec web pytest
```
## Useful Commands
```bash
# Restart containers
docker-compose restart
# Rebuild after requirements change
docker-compose down && docker-compose up -d --build
# View logs
docker-compose logs -f web
# Django shell
docker-compose exec web python manage.py shell
# Database shell
docker-compose exec db psql -U sportstime -d sportstime
```