Correct reporting of CAPEX / OPEX #302

lindnemi · 2024-12-09T14:14:04Z

n.statistics.capex and .opex do not actually report CAPEX and OPEX but fixed and marginal costs. Here we compute the Capex and Opex from the overnight_cost, lifetime, discount rate and fixed costs in the post networks. In the mid-term this should all be directly available from the PyPSA Network

Before asking for a review for this PR make sure to complete the following checklist:

Workflow with target rule ariadne_all completes without errors
The logic of export_ariadne_variables has been adapted to the changes
One or several figures that validate the changes in the PR have been posted as a comment
A brief description of the changes has been added to Changelog.md
The latest main has been merged into the PR
The config has a new prefix of the format YYYYMMDDdescriptive_title

for more information, see https://pre-commit.ci

github-actions · 2024-12-09T15:21:14Z

Validator Report

I am the Validator. Download all artifacts here.
I'll be back and edit this comment for each new commit.

Ariadne Variables

Comparison

	NRMSE	Pearson
System Cost\|OPEX\|Electricity\|Hydro	0.972	0.000
System Cost\|OPEX\|Electricity\|Storage Reservoir\|Hydro Dam Reservoir	0.938	0.000
System Cost\|OPEX\|Electricity\|Storage Converter\|Hydro Dam Reservoir	0.938	0.000
System Cost\|OPEX\|Electricity\|Storage Reservoir	0.935	0.000
System Cost\|OPEX\|Electricity\|Storage Converter	0.935	0.000
System Cost\|OPEX\|Heat\|Non-Renewable Waste	0.758	0.000
System Cost\|OPEX\|Electricity\|Non-Renewable Waste	0.758	0.996
System Cost\|OPEX\|Heat	0.747	-0.974
System Cost\|OPEX\|Electricity\|Gas\|OC	0.704	0.000
System Cost\|OPEX\|Electricity\|Solar\|PV\|Open Field	0.697	0.841
System Cost\|OPEX\|Heat\|Resistive heater	0.652	0.937
System Cost\|OPEX\|Electricity\|Wind\|Onshore	0.650	0.969
System Cost\|OPEX\|Heat\|Storage Reservoir	0.642	0.987
System Cost\|OPEX\|Heat\|Storage Converter	0.641	0.000
System Cost\|OPEX\|Electricity\|Solar\|PV	0.640	0.908
System Cost\|OPEX\|Electricity\|Solar	0.640	0.908
Investment\|Energy Supply\|Electricity\|Distribution	0.633	0.222
System Cost\|OPEX\|Electricity	0.629	-0.869
System Cost\|OPEX\|Electricity\|Storage Converter\|Pump Hydro	0.625	0.000
System Cost\|OPEX\|Electricity\|Storage Reservoir\|Pump Hydro	0.625	0.000
System Cost\|OPEX\|Electricity\|Wind	0.619	0.967
System Cost\|OPEX\|Heat\|Gas	0.616	-0.525
Investment\|Energy Supply\|Electricity\|Transmission and Distribution	0.612	0.418
System Cost\|OPEX\|Electricity\|Solar\|PV\|Rooftop	0.599	0.880
System Cost\|OPEX\|Liquids	0.593	0.984
System Cost\|OPEX\|Electricity\|Wind\|Offshore	0.592	0.961
System Cost\|OPEX\|Hydrogen	0.591	0.932
System Cost\|OPEX\|Heat\|Processes	0.590	0.000
System Cost\|OPEX\|Liquids\|Hydrogen	0.589	0.989
System Cost\|OPEX\|Electricity\|Storage Converter\|Stationary Batteries	0.577	0.000
System Cost\|OPEX\|Heat\|Hydrogen	0.577	0.000
System Cost\|OPEX\|Electricity\|Hydrogen	0.577	1.000
System Cost\|OPEX\|Electricity\|Hydrogen\|CC	0.577	1.000
Investment\|Energy Supply\|Electricity	0.553	0.713
System Cost\|OPEX\|Gases	0.552	0.916
System Cost\|OPEX\|Gases\|Biomass	0.552	0.916
System Cost\|OPEX\|Hydrogen\|Electricity	0.539	0.922
System Cost\|OPEX\|Electricity\|Gas	0.537	0.911
System Cost\|OPEX\|Electricity\|Gas\|w/o CCS	0.537	0.911
System Cost\|OPEX\|Heat\|Biomass\|w/ CCS	0.534	0.000
System Cost\|OPEX\|Heat\|Heat pump	0.519	0.939
System Cost\|OPEX\|Electricity\|Biomass	0.503	0.895
System Cost\|OPEX\|Electricity\|Biomass\|w/o CCS	0.503	0.895
System Cost\|OPEX\|Methanol	0.497	0.845
System Cost\|OPEX\|Hydrogen\|Gas	0.463	-0.042
System Cost\|OPEX\|Hydrogen\|Gas\|w/o CCS	0.463	-0.042
System Cost\|OPEX\|Electricity\|Gas\|CC	0.460	0.940
Capacity\|Electricity\|Storage Converter	0.458	0.370
System Cost\|OPEX\|Electricity\|Coal\|Lignite	0.452	0.838
System Cost\|OPEX\|Electricity\|Coal	0.448	0.913
System Cost\|OPEX\|Electricity\|Coal\|Hard Coal	0.441	0.979
System Cost\|OPEX\|Heat\|Renewable	0.438	-0.708
System Cost\|OPEX\|Electricity\|Biomass\|Solids	0.436	0.782
System Cost\|OPEX\|Heat\|Biomass	0.436	0.000
System Cost\|OPEX\|Heat\|Biomass\|w/o CCS	0.436	0.000
System Cost\|OPEX\|Electricity\|Oil	0.427	0.253
System Cost\|OPEX\|Electricity\|Gas\|w/ CCS	0.424	0.000
Secondary Energy\|Electricity\|Hydro	0.408	-0.513
System Cost\|OPEX\|Hydrogen\|Gas\|w/ CCS	0.408	0.000
System Cost\|OPEX\|Electricity\|Hydrogen\|OC	0.408	0.000
System Cost\|OPEX\|Electricity\|Biomass\|Gases and Liquids	0.408	0.789
System Cost\|OPEX\|Electricity\|Nuclear	0.408	1.000
System Cost\|OPEX\|Liquids\|Biomass	0.408	1.000
Price\|Secondary Energy\|Gases\|Hydrogen	0.294	0.000
Capacity\|Hydrogen\|Gas\|w/ CCS	0.227	0.851
Capacity Additions\|Hydrogen\|Gas\|w/ CCS	0.227	0.851
Capacity\|Electricity\|Gas\|OC	0.204	0.926
Investment\|Energy Supply\|Liquids\|Biomass	0.170	1.000
Price\|Secondary Energy\|Gases\|Biomass	0.134	0.000
Capacity Additions\|Liquids\|Biomass	0.118	1.000
Capacity\|Liquids\|Biomass	0.118	1.000
Investment\|Energy Supply\|Electricity\|Biomass\|w/o CCS	0.103	0.981
Investment\|Energy Supply\|Electricity\|Biomass	0.103	0.981
Investment\|Energy Supply\|Electricity\|Biomass\|Solids	0.103	0.981

NRMSE: Normalized (combined-min-max) Root Mean Square Error
Pearson: Pearson correlation coefficient
Threshold: NRMSE > 0.1
Only variables reaching the threshold are shown. Find the equivalent plot for all of them below.

Plots

Main branch	Feature branch

⚠️ Note: Only the first 20 variables are shown, but more are above the threshold. Find all of them in the artifacts.

General

Plots comparison

Main branch	Feature branch

Files comparison

	Status	NRMSE	MAE (norm)
csvs/market_values.csv	⚠️Changed	0.012	0.06
csvs/metrics.csv	⚠️Changed	0.000	0.07
csvs/price_statistics.csv	⚠️Changed	0.001	0.09
csvs/cumulative_cost.csv	⚠️Changed	0.000	0.09
csvs/nodal_supply_energy.csv	✅ Almost equal	0.000	0.01
csvs/nodal_cfs.csv	✅ Almost equal	0.012	0.00
csvs/prices.csv	✅ Almost equal	0.001	0.03
csvs/supply.csv	✅ Almost equal	0.001	0.00
csvs/capacities.csv	✅ Almost equal	0.000	0.00
csvs/nodal_costs.csv	✅ Almost equal	0.000	0.00
csvs/nodal_capacities.csv	✅ Almost equal	0.000	0.00
csvs/curtailment.csv	✅ Almost equal	0.000	0.05
csvs/cfs.csv	✅ Almost equal	0.038	0.00
csvs/costs.csv	✅ Almost equal	0.000	0.01
csvs/supply_energy.csv	✅ Almost equal	0.000	0.02
csvs/energy.csv	✅ Almost equal	0.000	0.01
csvs/weighted_prices.csv	✅ Equal

NRMSE: Normalized (combined-min-max) Root Mean Square Error
MAE (norm): Mean Absolute Error on normalized data (min-max)
Status Threshold: MAE (norm) > 0.05 and NRMSE > 0.3

Model Metrics

Benchmarks

Comparing report-CAPEX-OPEX (683a786) with main (5233ab0).
Branch is 13 commits ahead and 0 commits behind.
Last updated on 2024-12-12 11:58:03 CET.

for more information, see https://pre-commit.ci

lindnemi · 2024-12-10T15:51:51Z

At the moment grid CAPEX is only for assets built after 2020. How to harmonize this with the other sectors, which consider assets build before 2020 as well??

for more information, see https://pre-commit.ci

lindnemi · 2024-12-12T12:28:20Z

Current state of computing CAPEX:

CAPEX:

understood as annualized investments
WACC is 7%, but for decentral assest like solar rooftop it is 4%
For transmission infrastructure we only consider the annuities of assets built between 2020-2045, because older infrastructure likely is already amortized
For other infrastructure we would be able to consider all annuities. However for consistency we follow the way it is done for grid infrastructure
known issues:
- decentral biomass boiler have WACC 7% which is inconsistent
- Some assets built before 2020 likely have not yet repaid their annuities -> CAPEX may be underestimated
- There might be mismatches of the lifetime recorded in the network component and the one in the cost-data, affecting both the annuity and the FOM computation

OPEX:

FOM is computed as the difference of CAPEX and aggregated fixed costs, then added onto...
n.statistics.opex, which aggregates marginal cost
energy costs should be reported separately, see below
the opex of fossil generators in the model would correspond to fuel costs. since these generators are not covered by the exporter script, this is not an issue for us
differing from CAPEX, the FOM of existing assets before 2020 is included in the computation because they still need to be maintained
known issues:
- Offwind grid connection has FOM == 0%
- Some components remain in the network, even though they are no longer used (e.g. coal) -> FOM may be overestimated

Energy costs:

not reported at the moment
imported e-fuels and H2 should be included

toniseibold · 2024-12-13T14:52:46Z

workflow/scripts/export_ariadne_variables.py

@@ -5021,6 +5167,18 @@ def process_postnetworks(n, n_start, model_year, snakemake, costs):
    #     # The values  in p_nom_opt may already be discretized, here we make sure that
    #     # the same logic is applied to p_nom and p_nom_min
    #     n.links.loc[h2_links, attr] = n.links.loc[h2_links, attr].apply(_h2_lambda)
+    logger.info("Assing average Kernnetz cost to carrier H2 pipeline (Kernnetz)")


toniseibold · 2024-12-13T14:56:47Z

workflow/scripts/export_ariadne_variables.py


-    return _get_capacities(n, region, _f, cap_string="Capacity Additions|")
+    Calculate total investment, CAPEX, and OPEX in the given region.
+"""


Add docstring to function

lindnemi and others added 2 commits December 9, 2024 15:12

adding FOM rows to dataframe

011f8dc

[pre-commit.ci] auto fixes from pre-commit.com hooks

b4f07c6

for more information, see https://pre-commit.ci

lindnemi and others added 4 commits December 9, 2024 17:57

add OPEX computation

818711f

[pre-commit.ci] auto fixes from pre-commit.com hooks

4336a15

for more information, see https://pre-commit.ci

add grid variables

631adc0

[pre-commit.ci] auto fixes from pre-commit.com hooks

0415c86

for more information, see https://pre-commit.ci

lindnemi and others added 7 commits December 11, 2024 14:25

differentiationg temporal scope of investment, annuities and FOM

d706cc1

[pre-commit.ci] auto fixes from pre-commit.com hooks

48ea6bc

for more information, see https://pre-commit.ci

fix importing of the costs.csv

e3bfc2b

[pre-commit.ci] auto fixes from pre-commit.com hooks

93ec45e

for more information, see https://pre-commit.ci

correct FOM for special case of offwind generators

cef674f

[pre-commit.ci] auto fixes from pre-commit.com hooks

034d5c0

for more information, see https://pre-commit.ci

integrating Kernnetz changes from PR #305

683a786

lindnemi marked this pull request as ready for review December 12, 2024 12:27

lindnemi requested review from JulianGeis and toniseibold December 12, 2024 12:29

toniseibold approved these changes Dec 13, 2024

View reviewed changes

lindnemi merged commit b262b9f into main Dec 17, 2024
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Correct reporting of CAPEX / OPEX #302

Correct reporting of CAPEX / OPEX #302

lindnemi commented Dec 9, 2024 •

edited

Loading

github-actions bot commented Dec 9, 2024 •

edited

Loading

lindnemi commented Dec 10, 2024 •

edited

Loading

lindnemi commented Dec 12, 2024 •

edited

Loading

toniseibold Dec 13, 2024

toniseibold Dec 13, 2024

Correct reporting of CAPEX / OPEX #302

Correct reporting of CAPEX / OPEX #302

Conversation

lindnemi commented Dec 9, 2024 • edited Loading

github-actions bot commented Dec 9, 2024 • edited Loading

Validator Report

lindnemi commented Dec 10, 2024 • edited Loading

lindnemi commented Dec 12, 2024 • edited Loading

toniseibold Dec 13, 2024

Choose a reason for hiding this comment

toniseibold Dec 13, 2024

Choose a reason for hiding this comment

lindnemi commented Dec 9, 2024 •

edited

Loading

github-actions bot commented Dec 9, 2024 •

edited

Loading

lindnemi commented Dec 10, 2024 •

edited

Loading

lindnemi commented Dec 12, 2024 •

edited

Loading