Automated EIA baseline comparison

.github/workflows/eia-update-check.yml

@@ -0,0 +1,44 @@
+name: EIA update check
+
+on:
+  push:
+    branches: 
+       - master
+  workflow_dispatch: {}
+
+jobs:
+  eia-check:
+    runs-on: ubuntu-latest
+
+    env:
+      # EIA API key stored as a GitHub Actions secret
+      EIA_API_KEY: ${{ secrets.EIA_API_KEY }}
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Install package
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install ".[dev]"
+
+      - name: Run EIA baseline comparison (save output)
+        run: |
+          # Fail the step if the Python command fails, even when piped through tee
+          set -o pipefail
+          mkdir -p tests/eia_update
+          python -m scout.AEO_update_helpers.baseline_comparison --year 2025 2>&1 | tee tests/eia_update/eia_baseline_output.txt
+
+      - name: Upload artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: eia-baseline-results
+          path: |
+            ./tests/eia_update/
+            ./generated/log_*.txt

.gitignore

@@ -9,6 +9,9 @@ __pycache__/
 
 .venv
 
+# Local environment variables (do not commit API keys)
+.env
+
 # EIA AEO microtable data files #
 #################################
 
@@ -84,6 +87,8 @@ ehthumbs.db
 *.log
 *.sqlite
 log_*.txt
+tests/eia_update/
+tests/eia_update/**/*
 
 # Software/language support files #
 ###################################

docs/eia_update.rst

@@ -0,0 +1,42 @@
+.. _eia-update:
+
+EIA Updates
+=================
+
+Scout's residential and commercial baseline energy use is anchored to the
+U.S. Energy Information Administration (EIA) Annual Energy Outlook (AEO).
+When Scout is updated to a new AEO release, we use a small helper script to
+compare Scout's internal microsegment totals against the official AEO tables.
+
+The script lives in ``scout/AEO_update_helpers/baseline_comparison.py``.
+At a high level it:
+
+* loads Scout's processed AEO microsegments file (``mseg_res_com_cz.json``),
+* aggregates energy use by building class, fuel type, and end use,
+* queries the EIA AEO API for the matching time-series,
+* compares Scout vs. EIA year-by-year, and
+* prints summary tables showing any large differences.
+
+Running the EIA update check
+----------------------------
+
+1. **Create a local `.env` file (not committed to git)** in the project root
+   (the same folder as ``pyproject.toml``) with your EIA API key::
+
+      EIA_API_KEY=YOUR_REAL_KEY_HERE
+
+   You can request a free key from EIA at https://www.eia.gov/opendata/register.php.
+
+2. **Install Scout and its dependencies** into a virtual environment
+   (see :ref:`install-guide`).
+
+3. **From the project root, run the helper script**, for example::
+
+      python -m scout.AEO_update_helpers.baseline_comparison --year 2025
+
+   Use ``--verbose`` to see the underlying API calls and any combinations where
+   data are missing.
+
+The output includes per-combination comparisons and simple roll-up tables by
+building class and fuel type. These reports make it easier to confirm that
+Scout's baseline matches the chosen AEO reference.

docs/index.rst

@@ -26,6 +26,7 @@ Documentation Contents
    analysis_approach
    thermal_load_components
    ecm_reference
+   eia_update
 ..   maintenance
 
 

docs/overview.rst

@@ -44,6 +44,9 @@ Scout ECMs and their baseline markets
 * The :ref:`applicable baseline market <ecm-applicable-baseline-market>` for an ECM enumerates all of the :ref:`climate zones <json-climate_zone>`, :ref:`building types <json-bldg_type>`, :ref:`structure types <json-structure_type>`, :ref:`end uses <json-end_use>`, :ref:`fuel types <json-fuel_type>`, and :ref:`technologies <json-technology>` that are relevant to an ECM.
 * ECM definitions can include distinct cost, efficiency, and lifetime values for each climate zone, building type, fuel type, end use, and/or technology type; probability distributions may also be placed on ECM cost, efficiency, and lifetime inputs.
 
+See :ref:`eia-update` for details on how Scout aligns its baseline with the latest
+EIA Annual Energy Outlook.
+
 .. _overview-adoption:
 
 ECMs are adopted one of two ways

pyproject.toml

@@ -28,6 +28,7 @@ dependencies = [
     "xlsxwriter",
     "matplotlib",
     "backoff",
+    "python-dotenv",
     "pyyaml",
     "jsonschema"
 ]
@@ -38,4 +39,5 @@ dependencies = [
 dev = [
     "openpyxl",
     "flake8 >= 7.0",
+    "tabulate",
 ]