admin/SportstimeAPI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add Django web app, CloudKit sync, dashboard, and game_datetime_utc export

Browse Source 
Adds the full Django application layer on top of sportstime_parser:
- core: Sport, Team, Stadium, Game models with aliases and league structure
- scraper: orchestration engine, adapter, job management, Celery tasks
- cloudkit: CloudKit sync client, sync state tracking, sync jobs
- dashboard: staff dashboard for monitoring scrapers, sync, review queue
- notifications: email reports for scrape/sync results
- Docker setup for deployment (Dockerfile, docker-compose, entrypoint)

Game exports now use game_datetime_utc (ISO 8601 UTC) instead of
venue-local date+time strings, matching the canonical format used
by the iOS app.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Trey t
2026-02-19 14:04:27 -06:00
parent 4353d5943c
commit 63acf7accb
114 changed files with 13070 additions and 887 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
templates/admin/index.html Normal file
View File

@@ -0,0 +1,18 @@
{% extends "admin/index.html" %}
{% load i18n %}
{% block content %}
<div style="margin-bottom: 20px; padding: 15px; background: linear-gradient(135deg, #1a1a2e 0%, #16213e 100%); border-radius: 8px; display: flex; justify-content: space-between; align-items: center;">
<div>
<h2 style="color: #fff; margin: 0; font-size: 1.2rem;">
<span style="color: #4fc3f7;">Dashboard</span>
</h2>
<p style="color: #adb5bd; margin: 5px 0 0 0; font-size: 0.9rem;">View statistics, scraper status, sync status, and review queue</p>
</div>
<a href="{% url 'dashboard:index' %}" style="background: #4fc3f7; color: #1a1a2e; padding: 10px 20px; border-radius: 6px; text-decoration: none; font-weight: 600;">
Open Dashboard
</a>
</div>
{{ block.super }}
{% endblock %}

254
templates/base.html Normal file
View File

@@ -0,0 +1,254 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{% block title %}{{ title|default:'SportsTime' }}{% endblock %} - SportsTime</title>
<style>
* {
box-sizing: border-box;
margin: 0;
padding: 0;
}
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
background: #f5f7fa;
color: #333;
line-height: 1.6;
}
.navbar {
background: linear-gradient(135deg, #1a1a2e 0%, #16213e 100%);
padding: 1rem 2rem;
display: flex;
justify-content: space-between;
align-items: center;
box-shadow: 0 2px 10px rgba(0,0,0,0.1);
}
.navbar h1 {
color: #fff;
font-size: 1.5rem;
font-weight: 600;
}
.navbar h1 span {
color: #4fc3f7;
}
.navbar-links {
display: flex;
gap: 1.5rem;
}
.navbar-links a {
color: #adb5bd;
text-decoration: none;
font-weight: 500;
transition: color 0.2s;
}
.navbar-links a:hover, .navbar-links a.active {
color: #4fc3f7;
}
.container {
max-width: 1400px;
margin: 0 auto;
padding: 2rem;
}
.page-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 2rem;
}
.page-header h2 {
font-size: 1.75rem;
color: #1a1a2e;
}
.btn {
display: inline-block;
padding: 0.5rem 1rem;
border-radius: 6px;
text-decoration: none;
font-weight: 500;
cursor: pointer;
border: none;
transition: all 0.2s;
}
.btn-primary {
background: #4fc3f7;
color: #1a1a2e;
}
.btn-primary:hover {
background: #29b6f6;
}
.btn-success {
background: #28a745;
color: white;
}
.btn-danger {
background: #dc3545;
color: white;
}
.btn-secondary {
background: #6c757d;
color: white;
}
.card {
background: white;
border-radius: 12px;
box-shadow: 0 2px 8px rgba(0,0,0,0.08);
padding: 1.5rem;
margin-bottom: 1.5rem;
}
.card-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 1rem;
padding-bottom: 0.75rem;
border-bottom: 1px solid #eee;
}
.card-header h3 {
font-size: 1.1rem;
color: #1a1a2e;
}
.stat-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 1.5rem;
}
.stat-card {
background: white;
border-radius: 12px;
box-shadow: 0 2px 8px rgba(0,0,0,0.08);
padding: 1.5rem;
text-align: center;
}
.stat-card.primary {
background: linear-gradient(135deg, #4fc3f7 0%, #29b6f6 100%);
color: white;
}
.stat-card.success {
background: linear-gradient(135deg, #66bb6a 0%, #43a047 100%);
color: white;
}
.stat-card.warning {
background: linear-gradient(135deg, #ffca28 0%, #ffa000 100%);
color: #333;
}
.stat-card.danger {
background: linear-gradient(135deg, #ef5350 0%, #e53935 100%);
color: white;
}
.stat-card .stat-value {
font-size: 2.5rem;
font-weight: 700;
margin-bottom: 0.25rem;
}
.stat-card .stat-label {
font-size: 0.9rem;
opacity: 0.9;
}
.table {
width: 100%;
border-collapse: collapse;
}
.table th, .table td {
padding: 0.75rem;
text-align: left;
border-bottom: 1px solid #eee;
}
.table th {
font-weight: 600;
color: #666;
font-size: 0.85rem;
text-transform: uppercase;
}
.table tbody tr:hover {
background: #f8f9fa;
}
.badge {
display: inline-block;
padding: 0.25rem 0.5rem;
border-radius: 4px;
font-size: 0.75rem;
font-weight: 600;
}
.badge-success { background: #d4edda; color: #155724; }
.badge-warning { background: #fff3cd; color: #856404; }
.badge-danger { background: #f8d7da; color: #721c24; }
.badge-info { background: #d1ecf1; color: #0c5460; }
.badge-secondary { background: #e2e3e5; color: #383d41; }
.progress-bar {
height: 8px;
background: #e9ecef;
border-radius: 4px;
overflow: hidden;
}
.progress-bar .fill {
height: 100%;
border-radius: 4px;
transition: width 0.3s;
}
.grid-2 {
display: grid;
grid-template-columns: repeat(2, 1fr);
gap: 1.5rem;
}
.grid-3 {
display: grid;
grid-template-columns: repeat(3, 1fr);
gap: 1.5rem;
}
@media (max-width: 768px) {
.grid-2, .grid-3 {
grid-template-columns: 1fr;
}
.stat-grid {
grid-template-columns: repeat(2, 1fr);
}
}
.alert {
padding: 1rem;
border-radius: 8px;
margin-bottom: 1rem;
}
.alert-success { background: #d4edda; color: #155724; border-left: 4px solid #28a745; }
.alert-warning { background: #fff3cd; color: #856404; border-left: 4px solid #ffc107; }
.alert-danger { background: #f8d7da; color: #721c24; border-left: 4px solid #dc3545; }
.alert-info { background: #d1ecf1; color: #0c5460; border-left: 4px solid #17a2b8; }
.text-muted { color: #6c757d; }
.text-success { color: #28a745; }
.text-danger { color: #dc3545; }
.text-warning { color: #ffc107; }
.mb-0 { margin-bottom: 0; }
.mb-1 { margin-bottom: 0.5rem; }
.mb-2 { margin-bottom: 1rem; }
.mt-1 { margin-top: 0.5rem; }
.mt-2 { margin-top: 1rem; }
</style>
{% block extra_css %}{% endblock %}
</head>
<body>
<nav class="navbar">
<h1>Sports<span>Time</span></h1>
<div class="navbar-links">
<a href="{% url 'dashboard:index' %}" {% if request.resolver_match.url_name == 'index' %}class="active"{% endif %}>Dashboard</a>
<a href="{% url 'dashboard:stats' %}" {% if request.resolver_match.url_name == 'stats' %}class="active"{% endif %}>Stats</a>
<a href="{% url 'dashboard:scraper_status' %}" {% if request.resolver_match.url_name == 'scraper_status' %}class="active"{% endif %}>Scraper</a>
<a href="{% url 'dashboard:sync_status' %}" {% if request.resolver_match.url_name == 'sync_status' %}class="active"{% endif %}>Sync</a>
<a href="{% url 'dashboard:review_queue' %}" {% if request.resolver_match.url_name == 'review_queue' %}class="active"{% endif %}>Review</a>
<a href="{% url 'dashboard:export' %}" {% if request.resolver_match.url_name == 'export' %}class="active"{% endif %}>Export</a>
<a href="{% url 'admin:index' %}">Admin</a>
</div>
</nav>
<div class="container">
{% if messages %}
{% for message in messages %}
<div class="alert alert-{{ message.tags }}">{{ message }}</div>
{% endfor %}
{% endif %}
{% block content %}{% endblock %}
</div>
{% block extra_js %}{% endblock %}
</body>
</html>

176
templates/dashboard/export.html Normal file
View File

@@ -0,0 +1,176 @@
{% extends 'base.html' %}
{% block content %}
<div class="page-header">
<h2>Export Data</h2>
</div>
<!-- Overview Stats -->
<div class="stat-grid mb-2">
<div class="stat-card">
<div class="stat-value">{{ counts.teams }}</div>
<div class="stat-label">Teams</div>
</div>
<div class="stat-card">
<div class="stat-value">{{ counts.stadiums }}</div>
<div class="stat-label">Stadiums</div>
</div>
<div class="stat-card">
<div class="stat-value">{{ counts.games|floatformat:0 }}</div>
<div class="stat-label">Games</div>
</div>
<div class="stat-card">
<div class="stat-value">{{ counts.team_aliases }}</div>
<div class="stat-label">Team Aliases</div>
</div>
<div class="stat-card">
<div class="stat-value">{{ counts.stadium_aliases }}</div>
<div class="stat-label">Stadium Aliases</div>
</div>
</div>
<div class="grid-2">
<!-- Export Options -->
<div class="card">
<div class="card-header">
<h3>Export Options</h3>
</div>
<form id="exportForm" method="get" action="{% url 'dashboard:export_download' %}">
<div style="margin-bottom: 1.5rem;">
<label style="display: block; font-weight: 600; margin-bottom: 0.5rem;">Data Types</label>
<div style="display: grid; gap: 0.5rem;">
<label style="display: flex; align-items: center; gap: 0.5rem; cursor: pointer;">
<input type="checkbox" name="type" value="sports" checked>
<span>Sports</span>
<span class="text-muted" style="font-size: 0.85rem;">({{ counts.sports }} sports)</span>
</label>
<label style="display: flex; align-items: center; gap: 0.5rem; cursor: pointer;">
<input type="checkbox" name="type" value="league_structure" checked>
<span>League Structure</span>
<span class="text-muted" style="font-size: 0.85rem;">({{ counts.conferences }} conferences, {{ counts.divisions }} divisions)</span>
</label>
<label style="display: flex; align-items: center; gap: 0.5rem; cursor: pointer;">
<input type="checkbox" name="type" value="teams" checked>
<span>Teams</span>
<span class="text-muted" style="font-size: 0.85rem;">({{ counts.teams }} teams)</span>
</label>
<label style="display: flex; align-items: center; gap: 0.5rem; cursor: pointer;">
<input type="checkbox" name="type" value="stadiums" checked>
<span>Stadiums</span>
<span class="text-muted" style="font-size: 0.85rem;">({{ counts.stadiums }} stadiums)</span>
</label>
<label style="display: flex; align-items: center; gap: 0.5rem; cursor: pointer;">
<input type="checkbox" name="type" value="games" checked>
<span>Games</span>
<span class="text-muted" style="font-size: 0.85rem;">({{ counts.games }} games)</span>
</label>
<label style="display: flex; align-items: center; gap: 0.5rem; cursor: pointer;">
<input type="checkbox" name="type" value="team_aliases" checked>
<span>Team Aliases</span>
<span class="text-muted" style="font-size: 0.85rem;">({{ counts.team_aliases }} aliases)</span>
</label>
<label style="display: flex; align-items: center; gap: 0.5rem; cursor: pointer;">
<input type="checkbox" name="type" value="stadium_aliases" checked>
<span>Stadium Aliases</span>
<span class="text-muted" style="font-size: 0.85rem;">({{ counts.stadium_aliases }} aliases)</span>
</label>
</div>
</div>
<div style="margin-bottom: 1.5rem;">
<label style="display: block; font-weight: 600; margin-bottom: 0.5rem;">Filter by Sport</label>
<select name="sport" style="width: 100%; padding: 0.5rem; border: 1px solid #ddd; border-radius: 6px;">
<option value="">All Sports</option>
{% for sport in sports %}
<option value="{{ sport.code }}">{{ sport.short_name }} - {{ sport.name }}</option>
{% endfor %}
</select>
</div>
<div style="margin-bottom: 1.5rem;">
<label style="display: block; font-weight: 600; margin-bottom: 0.5rem;">Filter Games by Year</label>
<select name="year" style="width: 100%; padding: 0.5rem; border: 1px solid #ddd; border-radius: 6px;">
<option value="">All Years</option>
{% for year in years %}
<option value="{{ year }}">{{ year }}</option>
{% endfor %}
</select>
<span class="text-muted" style="font-size: 0.85rem; display: block; margin-top: 0.25rem;">All games played in this calendar year, regardless of season</span>
</div>
<button type="submit" class="btn btn-primary" style="width: 100%; padding: 0.75rem;">
Download Export
</button>
</form>
</div>
<!-- Export Files Info -->
<div class="card">
<div class="card-header">
<h3>Output Files</h3>
</div>
<div style="display: grid; gap: 1rem;">
<div style="padding: 1rem; background: #f8f9fa; border-radius: 8px;">
<strong style="color: #1a1a2e;">sports_canonical.json</strong>
<p class="text-muted mb-0" style="font-size: 0.85rem; margin-top: 0.25rem;">
Sport metadata: display names, SF Symbol icons, brand colors, and season month ranges.
</p>
</div>
<div style="padding: 1rem; background: #f8f9fa; border-radius: 8px;">
<strong style="color: #1a1a2e;">league_structure.json</strong>
<p class="text-muted mb-0" style="font-size: 0.85rem; margin-top: 0.25rem;">
League hierarchy: sports as leagues, conferences, and divisions with parent_id references.
</p>
</div>
<div style="padding: 1rem; background: #f8f9fa; border-radius: 8px;">
<strong style="color: #1a1a2e;">teams_canonical.json</strong>
<p class="text-muted mb-0" style="font-size: 0.85rem; margin-top: 0.25rem;">
Teams with canonical IDs, abbreviations, stadium references, and conference/division IDs.
</p>
</div>
<div style="padding: 1rem; background: #f8f9fa; border-radius: 8px;">
<strong style="color: #1a1a2e;">stadiums_canonical.json</strong>
<p class="text-muted mb-0" style="font-size: 0.85rem; margin-top: 0.25rem;">
Stadiums with coordinates, capacity, timezone, and primary team abbreviations.
</p>
</div>
<div style="padding: 1rem; background: #f8f9fa; border-radius: 8px;">
<strong style="color: #1a1a2e;">games_canonical.json</strong>
<p class="text-muted mb-0" style="font-size: 0.85rem; margin-top: 0.25rem;">
Games with local date/time (converted from UTC), team names, abbreviations, and canonical IDs.
</p>
</div>
<div style="padding: 1rem; background: #f8f9fa; border-radius: 8px;">
<strong style="color: #1a1a2e;">team_aliases.json</strong>
<p class="text-muted mb-0" style="font-size: 0.85rem; margin-top: 0.25rem;">
Historical team names and abbreviations with validity date ranges.
</p>
</div>
<div style="padding: 1rem; background: #f8f9fa; border-radius: 8px;">
<strong style="color: #1a1a2e;">stadium_aliases.json</strong>
<p class="text-muted mb-0" style="font-size: 0.85rem; margin-top: 0.25rem;">
Stadium name aliases (e.g., former naming rights) with validity dates.
</p>
</div>
</div>
</div>
</div>
<!-- Quick Export Buttons -->
<div class="card mt-2">
<div class="card-header">
<h3>Quick Export</h3>
</div>
<div style="display: flex; gap: 1rem; flex-wrap: wrap;">
<a href="{% url 'dashboard:export_download' %}?type=sports&type=league_structure&type=teams&type=stadiums&type=games&type=team_aliases&type=stadium_aliases" class="btn btn-success">
Export All Data
</a>
<a href="{% url 'dashboard:export_download' %}?type=sports&type=league_structure&type=teams&type=stadiums" class="btn btn-primary">
Export Structure Only (No Games)
</a>
<a href="{% url 'dashboard:export_download' %}?type=games" class="btn btn-secondary">
Export Games Only
</a>
</div>
</div>
{% endblock %}

188
templates/dashboard/index.html Normal file
View File

@@ -0,0 +1,188 @@
{% extends 'base.html' %}
{% block content %}
<div class="page-header">
<h2>Dashboard</h2>
<div>
<a href="{% url 'admin:index' %}" class="btn btn-secondary">Admin Panel</a>
</div>
</div>
<!-- Overview Stats -->
<div class="stat-grid">
<div class="stat-card primary">
<div class="stat-value">{{ sports_count }}</div>
<div class="stat-label">Active Sports</div>
</div>
<div class="stat-card">
<div class="stat-value">{{ teams_count }}</div>
<div class="stat-label">Teams</div>
</div>
<div class="stat-card">
<div class="stat-value">{{ stadiums_count }}</div>
<div class="stat-label">Stadiums</div>
</div>
<div class="stat-card success">
<div class="stat-value">{{ games_count|floatformat:0 }}</div>
<div class="stat-label">Games</div>
</div>
<div class="stat-card {% if pending_reviews > 0 %}warning{% endif %}">
<div class="stat-value">{{ pending_reviews }}</div>
<div class="stat-label">Pending Reviews</div>
</div>
</div>
<div class="grid-2 mt-2">
<!-- Sport Stats -->
<div class="card">
<div class="card-header">
<h3>Stats by Sport</h3>
</div>
<table class="table">
<thead>
<tr>
<th>Sport</th>
<th>Teams</th>
<th>Stadiums</th>
<th>Games</th>
<th>Reviews</th>
</tr>
</thead>
<tbody>
{% for stat in sport_stats %}
<tr>
<td><strong>{{ stat.sport.short_name }}</strong></td>
<td>{{ stat.teams }}</td>
<td>{{ stat.stadiums }}</td>
<td>{{ stat.games }}</td>
<td>
{% if stat.pending_reviews > 0 %}
<span class="badge badge-warning">{{ stat.pending_reviews }}</span>
{% else %}
<span class="text-muted">0</span>
{% endif %}
</td>
</tr>
{% empty %}
<tr>
<td colspan="5" class="text-muted">No sports configured</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<!-- Recent Scrape Jobs -->
<div class="card">
<div class="card-header">
<h3>Recent Scrape Jobs</h3>
<a href="{% url 'dashboard:scraper_status' %}" class="btn btn-primary">View All</a>
</div>
<table class="table">
<thead>
<tr>
<th>Sport</th>
<th>Status</th>
<th>Games</th>
<th>Time</th>
</tr>
</thead>
<tbody>
{% for job in recent_jobs %}
<tr>
<td>{{ job.config.sport.short_name }}</td>
<td>
{% if job.status == 'completed' %}
<span class="badge badge-success">Completed</span>
{% elif job.status == 'running' %}
<span class="badge badge-info">Running</span>
{% elif job.status == 'failed' %}
<span class="badge badge-danger">Failed</span>
{% else %}
<span class="badge badge-secondary">{{ job.status }}</span>
{% endif %}
</td>
<td>{{ job.games_found }}</td>
<td class="text-muted">{{ job.created_at|timesince }} ago</td>
</tr>
{% empty %}
<tr>
<td colspan="4" class="text-muted">No recent jobs</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
<div class="grid-2">
<!-- Recent Syncs -->
<div class="card">
<div class="card-header">
<h3>Recent CloudKit Syncs</h3>
<a href="{% url 'dashboard:sync_status' %}" class="btn btn-primary">View All</a>
</div>
<table class="table">
<thead>
<tr>
<th>Type</th>
<th>Status</th>
<th>Records</th>
<th>Time</th>
</tr>
</thead>
<tbody>
{% for sync in recent_syncs %}
<tr>
<td>{{ sync.sync_type }}</td>
<td>
{% if sync.status == 'completed' %}
<span class="badge badge-success">Completed</span>
{% elif sync.status == 'running' %}
<span class="badge badge-info">Running</span>
{% elif sync.status == 'failed' %}
<span class="badge badge-danger">Failed</span>
{% else %}
<span class="badge badge-secondary">{{ sync.status }}</span>
{% endif %}
</td>
<td>{{ sync.records_synced }}</td>
<td class="text-muted">{{ sync.created_at|timesince }} ago</td>
</tr>
{% empty %}
<tr>
<td colspan="4" class="text-muted">No recent syncs</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<!-- Quick Actions -->
<div class="card">
<div class="card-header">
<h3>Quick Actions</h3>
</div>
<div style="display: grid; gap: 1rem;">
<a href="{% url 'admin:core_game_changelist' %}" class="btn btn-primary" style="text-align: center;">
Manage Games
</a>
<a href="{% url 'admin:core_team_changelist' %}" class="btn btn-primary" style="text-align: center;">
Manage Teams
</a>
<a href="{% url 'admin:core_stadium_changelist' %}" class="btn btn-primary" style="text-align: center;">
Manage Stadiums
</a>
<a href="{% url 'dashboard:review_queue' %}" class="btn {% if pending_reviews > 0 %}btn-danger{% else %}btn-secondary{% endif %}" style="text-align: center;">
Review Queue {% if pending_reviews > 0 %}({{ pending_reviews }}){% endif %}
</a>
<form method="post" action="{% url 'dashboard:run_sync' %}">
{% csrf_token %}
<button type="submit" class="btn btn-success" style="width: 100%;">
Trigger CloudKit Sync
</button>
</form>
</div>
</div>
</div>
{% endblock %}

96
templates/dashboard/review_queue.html Normal file
View File

@@ -0,0 +1,96 @@
{% extends 'base.html' %}
{% block content %}
<div class="page-header">
<h2>Review Queue</h2>
<span class="badge {% if total_pending > 0 %}badge-warning{% else %}badge-success{% endif %}" style="font-size: 1rem; padding: 0.5rem 1rem;">
{{ total_pending }} Pending
</span>
</div>
{% if total_pending == 0 %}
<div class="alert alert-success">
No items pending review. All data has been validated.
</div>
{% endif %}
<!-- Summary by Sport and Type -->
{% if review_summary %}
<div class="card">
<div class="card-header">
<h3>Review Summary</h3>
</div>
<div class="stat-grid">
{% for item in review_summary %}
<div class="stat-card warning">
<div class="stat-value">{{ item.count }}</div>
<div class="stat-label">{{ item.sport__short_name }} - {{ item.item_type|title }}</div>
</div>
{% endfor %}
</div>
</div>
{% endif %}
<!-- Pending Items -->
<div class="card">
<div class="card-header">
<h3>Pending Reviews</h3>
<a href="{% url 'admin:scraper_manualreviewitem_changelist' %}?status__exact=pending" class="btn btn-primary">View in Admin</a>
</div>
<table class="table">
<thead>
<tr>
<th>Type</th>
<th>Sport</th>
<th>Raw Value</th>
<th>Matched To</th>
<th>Confidence</th>
<th>Created</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for item in pending_items %}
<tr>
<td>
{% if item.item_type == 'team' %}
<span class="badge badge-info">Team</span>
{% elif item.item_type == 'stadium' %}
<span class="badge badge-secondary">Stadium</span>
{% elif item.item_type == 'game' %}
<span class="badge badge-warning">Game</span>
{% else %}
<span class="badge badge-secondary">{{ item.item_type }}</span>
{% endif %}
</td>
<td>{{ item.sport.short_name }}</td>
<td><strong>{{ item.raw_value }}</strong></td>
<td>
{% if item.matched_value %}
{{ item.matched_value }}
{% else %}
<span class="text-muted">No match</span>
{% endif %}
</td>
<td>
<div class="progress-bar" style="width: 80px;">
<div class="fill" style="width: {{ item.confidence }}%; background: {% if item.confidence >= 85 %}#28a745{% elif item.confidence >= 70 %}#ffc107{% else %}#dc3545{% endif %};"></div>
</div>
<small class="text-muted">{{ item.confidence }}%</small>
</td>
<td class="text-muted">{{ item.created_at|timesince }} ago</td>
<td>
<a href="{% url 'admin:scraper_manualreviewitem_change' item.id %}" class="btn btn-primary" style="padding: 0.25rem 0.5rem; font-size: 0.85rem;">
Review
</a>
</td>
</tr>
{% empty %}
<tr>
<td colspan="7" class="text-muted">No pending reviews</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endblock %}

256
templates/dashboard/scraper_status.html Normal file
View File

@@ -0,0 +1,256 @@
{% extends 'base.html' %}
{% block content %}
<div class="page-header">
<h2>Scraper Status</h2>
</div>
<!-- Status Overview -->
<div class="stat-grid mb-2">
<div class="stat-card {% if running_jobs > 0 %}primary{% endif %}">
<div class="stat-value">{{ running_jobs }}</div>
<div class="stat-label">Running Jobs</div>
</div>
<div class="stat-card {% if pending_jobs > 0 %}warning{% endif %}">
<div class="stat-value">{{ pending_jobs }}</div>
<div class="stat-label">Pending Jobs</div>
</div>
</div>
<!-- Scraper Configs -->
<div class="card">
<div class="card-header" style="display: flex; justify-content: space-between; align-items: center;">
<h3 style="margin: 0;">Scraper Configurations</h3>
<form method="post" action="{% url 'dashboard:run_all_scrapers' %}" style="display: inline;">
{% csrf_token %}
<button type="submit" class="btn btn-primary">Run All Enabled</button>
</form>
</div>
<table class="table">
<thead>
<tr>
<th>Sport</th>
<th>Season</th>
<th>Status</th>
<th>Last Run</th>
<th>Games Found</th>
<th>Source</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for config in configs %}
<tr>
<td><strong>{{ config.sport.short_name }}</strong></td>
<td>{{ config.season }}</td>
<td>
{% if config.is_enabled %}
<span class="badge badge-success">Active</span>
{% else %}
<span class="badge badge-secondary">Inactive</span>
{% endif %}
</td>
<td>
{% if config.last_run %}
{{ config.last_run|timesince }} ago
{% if config.last_run_status == 'completed' %}
<span class="text-success">✓</span>
{% elif config.last_run_status == 'failed' %}
<span class="text-danger">✗</span>
{% endif %}
{% else %}
<span class="text-muted">Never</span>
{% endif %}
</td>
<td>
{% if config.last_run_games %}
{{ config.last_run_games }} games
{% else %}
<span class="text-muted">-</span>
{% endif %}
</td>
<td>{{ config.primary_source|default:"auto" }}</td>
<td>
<form method="post" action="{% url 'dashboard:run_scraper' config.sport.code config.season %}" style="display: inline;">
{% csrf_token %}
<button type="submit" class="btn btn-primary" style="padding: 0.25rem 0.5rem; font-size: 0.85rem;">
Run Now
</button>
</form>
</td>
</tr>
{% empty %}
<tr>
<td colspan="7" class="text-muted">No scraper configurations</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<!-- Recent Jobs -->
<div class="card">
<div class="card-header">
<h3>Recent Jobs</h3>
</div>
<table class="table">
<thead>
<tr>
<th>Sport</th>
<th>Season</th>
<th>Status</th>
<th>Games</th>
<th>Started</th>
<th>Duration</th>
<th>Error</th>
</tr>
</thead>
<tbody>
{% for job in recent_jobs %}
<tr>
<td><strong>{{ job.config.sport.short_name }}</strong></td>
<td>{{ job.config.season }}</td>
<td>
{% if job.status == 'completed' %}
<span class="badge badge-success">Completed</span>
{% elif job.status == 'running' %}
<span class="badge badge-info">Running</span>
{% elif job.status == 'failed' %}
<span class="badge badge-danger">Failed</span>
{% elif job.status == 'pending' %}
<span class="badge badge-warning">Pending</span>
{% else %}
<span class="badge badge-secondary">{{ job.status }}</span>
{% endif %}
</td>
<td>
{% if job.status == 'completed' %}
{{ job.games_found }} found, +{{ job.games_new }} new
{% else %}
-
{% endif %}
</td>
<td class="text-muted">{{ job.created_at|timesince }} ago</td>
<td>{{ job.duration_display }}</td>
<td>
{% if job.error_message %}
<span class="text-danger" title="{{ job.error_message }}">{{ job.error_message|truncatechars:50 }}</span>
{% else %}
-
{% endif %}
</td>
</tr>
{% empty %}
<tr>
<td colspan="7" class="text-muted">No recent jobs</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<!-- Documentation -->
<div class="card">
<div class="card-header">
<h3>How Scrapers Work</h3>
</div>
<div style="padding: 1rem;">
<h4 style="margin-top: 0;">What Gets Updated Automatically</h4>
<p>When a scraper runs, it fetches schedule data from official sources and updates the following:</p>
<table class="table" style="margin-bottom: 1.5rem;">
<thead>
<tr>
<th>Data Type</th>
<th>Behavior</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Games</strong></td>
<td>Creates new games, updates scores/status for existing games. Uses canonical IDs to match.</td>
</tr>
<tr>
<td><strong>Teams</strong></td>
<td>Auto-created from scraper's built-in team mappings. New teams are added automatically.</td>
</tr>
<tr>
<td><strong>Stadiums</strong></td>
<td>Auto-created from scraper's built-in stadium mappings. New venues are added automatically.</td>
</tr>
<tr>
<td><strong>Conferences</strong></td>
<td>Auto-created based on team data (e.g., Eastern, Western).</td>
</tr>
<tr>
<td><strong>Divisions</strong></td>
<td>Auto-created based on team data (e.g., Atlantic, Pacific).</td>
</tr>
</tbody>
</table>
<h4>New Team Scenario</h4>
<p>If a league adds a new team (e.g., expansion team):</p>
<ol>
<li>Add the team via <a href="/admin/core/team/add/">Admin → Teams</a></li>
<li>Add <a href="/admin/core/teamalias/add/">Team Aliases</a> for any names/abbreviations used by data sources</li>
<li>Add the stadium via <a href="/admin/core/stadium/add/">Admin → Stadiums</a> (if it's a new venue)</li>
<li>Add <a href="/admin/core/stadiumalias/add/">Stadium Aliases</a> for any alternate names used by data sources</li>
<li>Run the scraper - it will automatically import all the new team's games</li>
</ol>
<p class="text-muted" style="font-size: 0.9rem;">If the scraper encounters an unknown team or stadium name, it creates a <strong>Review Item</strong> for manual resolution.</p>
<h4>What Requires Manual Action</h4>
<table class="table" style="margin-bottom: 1.5rem;">
<thead>
<tr>
<th>Situation</th>
<th>Action Required</th>
</tr>
</thead>
<tbody>
<tr>
<td>Unknown team name in schedule</td>
<td>Add a <a href="/admin/core/teamalias/add/">Team Alias</a> in the admin, or resolve in <a href="{% url 'dashboard:review_queue' %}">Review Queue</a></td>
</tr>
<tr>
<td>Unknown stadium name</td>
<td>Add a <a href="/admin/core/stadiumalias/add/">Stadium Alias</a> in the admin, or resolve in Review Queue</td>
</tr>
<tr>
<td>New expansion team</td>
<td>Add a <a href="/admin/core/team/add/">new Team</a> in the admin, then add aliases for any alternate names</td>
</tr>
<tr>
<td>Team relocation/rename</td>
<td>Add a <a href="/admin/core/teamalias/add/">Team Alias</a> with validity dates for the old name</td>
</tr>
<tr>
<td>Stadium rename (naming rights)</td>
<td>Add a <a href="/admin/core/stadiumalias/add/">Stadium Alias</a> with validity dates (e.g., "Staples Center" valid until 2021)</td>
</tr>
</tbody>
</table>
<h4>Managing Aliases via Admin</h4>
<p>Team and stadium name mappings can be managed directly in the admin interface:</p>
<ul>
<li><a href="/admin/core/teamalias/">Team Aliases</a> - Map alternate team names, abbreviations, historical names</li>
<li><a href="/admin/core/stadiumalias/">Stadium Aliases</a> - Map alternate stadium names, former names (naming rights changes)</li>
</ul>
<p>Aliases support <strong>validity dates</strong> - useful for historical names like "Washington Redskins" (valid until 2020) or stadium naming rights changes.</p>
<h4>Data Flow</h4>
<p style="font-family: monospace; background: #f5f5f5; padding: 0.75rem; border-radius: 4px;">
Scraper runs → Fetches from source (ESPN, league API, etc.) → Normalizes team/stadium names → Creates/updates records → Marks changed records for CloudKit sync → Creates review items for unresolved names
</p>
<h4>Tips</h4>
<ul style="margin-bottom: 0;">
<li>Run scrapers regularly to keep scores and game statuses current</li>
<li>Check the <a href="{% url 'dashboard:review_queue' %}">Review Queue</a> after scrapes for items needing attention</li>
<li>Scrapers are idempotent - running multiple times is safe and won't duplicate data</li>
<li>Each sport uses multiple data sources with automatic fallback if one fails</li>
</ul>
</div>
</div>
{% endblock %}

111
templates/dashboard/stats.html Normal file
View File

@@ -0,0 +1,111 @@
{% extends 'base.html' %}
{% block content %}
<div class="page-header">
<h2>Statistics</h2>
</div>
<!-- Game Stats -->
<div class="card">
<div class="card-header">
<h3>Game Statistics</h3>
</div>
<div class="stat-grid">
<div class="stat-card">
<div class="stat-value">{{ game_stats.total|floatformat:0 }}</div>
<div class="stat-label">Total Games</div>
</div>
<div class="stat-card success">
<div class="stat-value">{{ game_stats.final|floatformat:0 }}</div>
<div class="stat-label">Completed</div>
</div>
<div class="stat-card primary">
<div class="stat-value">{{ game_stats.scheduled|floatformat:0 }}</div>
<div class="stat-label">Scheduled</div>
</div>
<div class="stat-card warning">
<div class="stat-value">{{ game_stats.today }}</div>
<div class="stat-label">Today</div>
</div>
<div class="stat-card">
<div class="stat-value">{{ game_stats.this_week }}</div>
<div class="stat-label">This Week</div>
</div>
</div>
</div>
<!-- Sync Stats -->
<div class="card">
<div class="card-header">
<h3>CloudKit Sync Statistics</h3>
</div>
<div class="stat-grid">
<div class="stat-card">
<div class="stat-value">{{ sync_stats.total|floatformat:0 }}</div>
<div class="stat-label">Total Records</div>
</div>
<div class="stat-card success">
<div class="stat-value">{{ sync_stats.synced|floatformat:0 }}</div>
<div class="stat-label">Synced</div>
</div>
<div class="stat-card warning">
<div class="stat-value">{{ sync_stats.pending|floatformat:0 }}</div>
<div class="stat-label">Pending</div>
</div>
<div class="stat-card danger">
<div class="stat-value">{{ sync_stats.failed|floatformat:0 }}</div>
<div class="stat-label">Failed</div>
</div>
</div>
{% if sync_stats.total > 0 %}
<div class="mt-2">
<div style="display: flex; justify-content: space-between; margin-bottom: 0.5rem;">
<span>Sync Progress</span>
<span>{{ sync_stats.synced }} / {{ sync_stats.total }} ({% widthratio sync_stats.synced sync_stats.total 100 %}%)</span>
</div>
<div class="progress-bar">
<div class="fill" style="width: {% widthratio sync_stats.synced sync_stats.total 100 %}%; background: #28a745;"></div>
</div>
</div>
{% endif %}
</div>
<!-- Sport Breakdown -->
<div class="card">
<div class="card-header">
<h3>Statistics by Sport</h3>
</div>
<table class="table">
<thead>
<tr>
<th>Sport</th>
<th>Teams</th>
<th>Stadiums</th>
<th>Games</th>
<th>Pending Reviews</th>
</tr>
</thead>
<tbody>
{% for stat in sport_stats %}
<tr>
<td>
<strong>{{ stat.sport.short_name }}</strong>
<div class="text-muted" style="font-size: 0.8rem;">{{ stat.sport.name }}</div>
</td>
<td>{{ stat.teams }}</td>
<td>{{ stat.stadiums }}</td>
<td>{{ stat.games }}</td>
<td>
{% if stat.pending_reviews > 0 %}
<span class="badge badge-warning">{{ stat.pending_reviews }}</span>
{% else %}
<span class="badge badge-success">0</span>
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endblock %}