v0.2.0

[grgmiller]

Release Notes

2021 Data Release

• This release includes new data for 2021, and updates the year 2019 and 2020 data.

Hourly data for all individual plants

• The plant data results now include hourly data for every individual plant in the U.S., including those plants for which we impute the hourly generation profile. Previously, we had aggregated the imputed data to the fleet level. Details Add hourly data for all individual plants #246

Updates OGE pipeline to work with PUDL v2022.11.30

• Updates the pipeline to work with v2022.11.30 of PUDL, which introduced many breaking changes. See Update OGE to work with the next PUDL release / 2021 data #258
• The new PUDL release now performs many of the CEMS data cleaning steps that we previously performed in the OGE pipeline, so these data cleaning steps have been removed

Fixes a bug that was overestimating NOx and SO2 emissions for some plants

• Some NOx and SO2 control data is missing control ID numbers in EIA-923, which was causing this data to get dropped, which meant that OGE was treating emissions from these generators as uncontrolled. This update fixes that issue. See Missing emissions control IDs leading to overestimation of NOX emissions #255

Patches bugs with consumed emissions calculation

• Updates environment to fix bug that was leading to random missing values in consumed calculation on some operating systems.
• When calculating consumed emissions, we calculate demand in a BA by subtracting interchange from generation, but for certain BAs, this approach results in negative emission factors being calculated, we we directly use reported demand from EIA-930. We updated the list of BAs for which we apply this approach, which is now differentiated by year so that we are only performing this patch where strictly necessary.
• Makes the approach for identifying imputed values in the EIA-930 data cleaned by gridemissions consistent across the hourly shaping step and the consumed emissions step of the data pipeline.
• Updates the manual cleaning of EIA-930 data to remove OVEC corrections, and add a timestamp offset for SC prior to 12-31-2020

Validation

• Add a validation check for missing values in all results files

Output files

• Export a cleaned version of the unit-level CEMS dataset to outputs (outputs/cems_cleaned.csv). Previously we only exported a version after aggregation to subplants and gross-to-net-generation conversion. This original file was renamed outputs/cems_subplant.csv.
• Add an option to export metric files or not when outputting data

Balancing authority updates

• Anchorage Municipal Light and Power retired on October 30, 2020
• Electric Energy, Inc (EEI) changed to a generation-only BA
• Map Pacific Gas and Electric utility to CISO

Emissions Calculation Updates

• When imputing missing emissions data in CEMS, we now calculated fuel-weighted emission factors for each subplant-month which are used for imputing missing emissions values. This is based on the total consumption of each type of fuel that is reported to be consumed in each subplant-month in EIA-923. The process for imputing missing emissions is now:

1. If a unit has non-missing emissions data for other hours in the same month, calculate a unit-month specific EF from the CEMS-reported fuel consumption and emissions
2. For all remaining missing values, use the subplant and month-specific weighted average emission factor from subplant_emission_factors calculated from the EIA-923 data
3. For any remaining missing values, calculate emissions based on the subplant primary fuel and fuel consumption
4. For any remaining missing values, calculate emissions based on the fuel type assigned in the power sector data crosswalk.

• Previously, when assigning a fuel type to each emissions_unit_id_epa, we prioritized using the fuel type reported in the power sector data crosswalk. We now identify the primary fuel type for each subplant using EIA-923 fuel consumption data, and use this to assign a fuel type to each CEMS unit. The fuel type reported in the power sector data crosswalk is now used as a last case.
• Add a NOx emission factor for CS prime movers with OG fuel consumption

Subplant identification

• Manually assign all units at plant 1391 to a single subplant

EPA-EIA Crosswalk

• Update to use v0.3 of the power sector data crosswalk
• The crosswalk is now integrated as a table into PUDL, so we will use the pudl cleaned version in the pipeline
• Add manual unit to generator crosswalks for plants 60925, 60910, 63259
• Identified plant 59073 (Cove Point LNG Terminal) as a non-grid connected plant

Notebooks

• Add notebook to explore the reported fuel heat content for each fuel (notebooks/manual_data/export_fuel_heat_content.ipynb)
• Update notebook used to identify uncorrected time lags in raw EIA-930 data

Known issues

• The gross to net conversion for two plants (plant 55799 in 2019, and plant 57865 in 2021) is likely incorrect due to data inconsistencies between reported gross generation in CEMS and reported net generation in EIA-923 for these plants.

What's Changed

• Integrate Power Sector Data Crosswalk v0.3 by @grgmiller in Integrate Power Sector Data Crosswalk v0.3 #244
• Fix missing NOx and SO2 identifiers by @grgmiller in Fix missing NOx and SO2 identifiers #256
• Add hourly data for all individual plants by @grgmiller in Add hourly data for all individual plants #246
• Gailin/evaluate 2021 lags by @gailin-p in Gailin/evaluate 2021 lags #264
• Patch negative efs by @grgmiller in Patch negative efs #265
• Fix consumed calc by @gailin-p in Fix consumed calc #266
• Update OGE for to work with PUDL v.2022.11.30 and integrate 2021 data by @grgmiller in Update OGE for to work with PUDL v.2022.11.30 and integrate 2021 data #259
• Release 0.2.0 by @grgmiller in Release 0.2.0 #267

Full Changelog: v0.1.2...v0.2.0

---

This discussion was created from the release v0.2.0.