{% extends 'base.html' %} {% block content %}
{{ running_jobs }}
Running Jobs
{{ pending_jobs }}
Pending Jobs

Scraper Configurations

{% csrf_token %}
{% for config in configs %} {% empty %} {% endfor %}
Sport Season Status Last Run Games Found Source Actions
{{ config.sport.short_name }} {{ config.season }} {% if config.is_enabled %} Active {% else %} Inactive {% endif %} {% if config.last_run %} {{ config.last_run|timesince }} ago {% if config.last_run_status == 'completed' %} {% elif config.last_run_status == 'failed' %} {% endif %} {% else %} Never {% endif %} {% if config.last_run_games %} {{ config.last_run_games }} games {% else %} - {% endif %} {{ config.primary_source|default:"auto" }}
{% csrf_token %}
No scraper configurations

Recent Jobs

{% for job in recent_jobs %} {% empty %} {% endfor %}
Sport Season Status Games Started Duration Error
{{ job.config.sport.short_name }} {{ job.config.season }} {% if job.status == 'completed' %} Completed {% elif job.status == 'running' %} Running {% elif job.status == 'failed' %} Failed {% elif job.status == 'pending' %} Pending {% else %} {{ job.status }} {% endif %} {% if job.status == 'completed' %} {{ job.games_found }} found, +{{ job.games_new }} new {% else %} - {% endif %} {{ job.created_at|timesince }} ago {{ job.duration_display }} {% if job.error_message %} {{ job.error_message|truncatechars:50 }} {% else %} - {% endif %}
No recent jobs

How Scrapers Work

What Gets Updated Automatically

When a scraper runs, it fetches schedule data from official sources and updates the following:

Data Type Behavior
Games Creates new games, updates scores/status for existing games. Uses canonical IDs to match.
Teams Auto-created from scraper's built-in team mappings. New teams are added automatically.
Stadiums Auto-created from scraper's built-in stadium mappings. New venues are added automatically.
Conferences Auto-created based on team data (e.g., Eastern, Western).
Divisions Auto-created based on team data (e.g., Atlantic, Pacific).

New Team Scenario

If a league adds a new team (e.g., expansion team):

  1. Add the team via Admin → Teams
  2. Add Team Aliases for any names/abbreviations used by data sources
  3. Add the stadium via Admin → Stadiums (if it's a new venue)
  4. Add Stadium Aliases for any alternate names used by data sources
  5. Run the scraper - it will automatically import all the new team's games

If the scraper encounters an unknown team or stadium name, it creates a Review Item for manual resolution.

What Requires Manual Action

Situation Action Required
Unknown team name in schedule Add a Team Alias in the admin, or resolve in Review Queue
Unknown stadium name Add a Stadium Alias in the admin, or resolve in Review Queue
New expansion team Add a new Team in the admin, then add aliases for any alternate names
Team relocation/rename Add a Team Alias with validity dates for the old name
Stadium rename (naming rights) Add a Stadium Alias with validity dates (e.g., "Staples Center" valid until 2021)

Managing Aliases via Admin

Team and stadium name mappings can be managed directly in the admin interface:

Aliases support validity dates - useful for historical names like "Washington Redskins" (valid until 2020) or stadium naming rights changes.

Data Flow

Scraper runs → Fetches from source (ESPN, league API, etc.) → Normalizes team/stadium names → Creates/updates records → Marks changed records for CloudKit sync → Creates review items for unresolved names

Tips

{% endblock %}