Adds the full Django application layer on top of sportstime_parser: - core: Sport, Team, Stadium, Game models with aliases and league structure - scraper: orchestration engine, adapter, job management, Celery tasks - cloudkit: CloudKit sync client, sync state tracking, sync jobs - dashboard: staff dashboard for monitoring scrapers, sync, review queue - notifications: email reports for scrape/sync results - Docker setup for deployment (Dockerfile, docker-compose, entrypoint) Game exports now use game_datetime_utc (ISO 8601 UTC) instead of venue-local date+time strings, matching the canonical format used by the iOS app. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
4.0 KiB
4.0 KiB
CLAUDE.md
This file provides context for Claude Code when working on this project.
Project Overview
SportsTime is a Django-based sports data pipeline that scrapes game schedules from official sources, normalizes the data, stores it in PostgreSQL, and syncs to CloudKit for iOS app consumption.
Architecture
┌─────────────────┐ ┌──────────────┐ ┌─────────────┐ ┌──────────┐
│ Data Sources │ ──▶ │ Scrapers │ ──▶ │ PostgreSQL │ ──▶ │ CloudKit │
│ (ESPN, leagues) │ │ (sportstime_ │ │ (Django) │ │ (iOS) │
└─────────────────┘ │ parser) │ └─────────────┘ └──────────┘
└──────────────┘
Key Directories
core/- Django models: Sport, Team, Stadium, Game, Conference, Division, Aliasesscraper/- Scraper orchestration, adapter, job managementsportstime_parser/- Standalone scraper library (ESPN, league APIs)cloudkit/- CloudKit sync client and job managementdashboard/- Staff dashboard for monitoring and controlstemplates/- Django templates for dashboard UI
Data Flow
- Scraper runs (manual or scheduled via Celery Beat)
- sportstime_parser fetches from ESPN/league APIs
- Adapter normalizes data and resolves team/stadium names
- Django models store normalized data with CloudKit sync flags
- CloudKit sync pushes pending records to iCloud
Models Hierarchy
Sport
├── Conference
│ └── Division
│ └── Team (has TeamAliases)
├── Stadium (has StadiumAliases)
└── Game (references Team, Stadium)
Name Resolution
Team and stadium names from scraped data are resolved via:
- Direct ID match (canonical IDs from scraper)
- Database aliases (TeamAlias/StadiumAlias with date validity)
- Direct name/abbreviation match
Aliases support validity dates for historical names (e.g., team relocations, stadium naming rights).
Common Tasks
Run a scraper
docker-compose exec web python manage.py shell
>>> from scraper.tasks import run_scraper_task
>>> run_scraper_task.delay(config_id)
Check scraper status
Visit /dashboard/scraper-status/ or check ScrapeJob model.
Add team/stadium alias
Use Django admin at /admin/core/teamalias/ or /admin/core/stadiumalias/.
Export/Import data
All admin models support import/export (JSON, CSV, XLSX) via django-import-export.
Sync to CloudKit
docker-compose exec web python manage.py shell
>>> from cloudkit.tasks import run_cloudkit_sync
>>> run_cloudkit_sync.delay(config_id)
Environment
- Docker Compose for local development
- PostgreSQL database
- Redis for Celery broker
- Celery for async tasks and scheduled jobs
Key Files
sportstime/settings.py- Django settingsscraper/engine/adapter.py- Bridges sportstime_parser to Djangoscraper/engine/db_alias_loader.py- Database alias resolutioncore/resources.py- Import/export resource definitionsdocker-compose.yml- Container orchestration
Supported Sports
| Code | Sport | Season Type |
|---|---|---|
| nba | NBA Basketball | split (Oct-Jun) |
| mlb | MLB Baseball | calendar (Mar-Oct) |
| nfl | NFL Football | split (Sep-Feb) |
| nhl | NHL Hockey | split (Oct-Jun) |
| mls | MLS Soccer | calendar (Feb-Nov) |
| wnba | WNBA Basketball | calendar (May-Sep) |
| nwsl | NWSL Soccer | calendar (Mar-Nov) |
Testing
docker-compose exec web pytest
Useful Commands
# Restart containers
docker-compose restart
# Rebuild after requirements change
docker-compose down && docker-compose up -d --build
# View logs
docker-compose logs -f web
# Django shell
docker-compose exec web python manage.py shell
# Database shell
docker-compose exec db psql -U sportstime -d sportstime