Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify dh systems #113

Merged
merged 19 commits into from
Aug 14, 2024
Merged

Modify dh systems #113

merged 19 commits into from
Aug 14, 2024

Conversation

cpschau
Copy link
Contributor

@cpschau cpschau commented Jun 17, 2024

For a more accurate representation of the district heating sector in Germany, this PR integrates the DemandRegio data from the open ego project. The data is resolved on NUTS3 level and provides for each of the regions the household count for different heating technologies.
Egon data:
nuts3_DH-share
The count for households with access to district heating networks is used to correct the district heating and urban shares of the clusters calculated in build_district_heat_share.

dh_shares_2020_pre dh_shares_2020_post
dh_shares_2030_pre dh_shares_2030_post
dh_shares_2045_pre dh_shares_2045_post

@cpschau cpschau marked this pull request as ready for review August 6, 2024 12:39
@cpschau cpschau force-pushed the modify_dh_systems branch from 6dcc649 to 27a2369 Compare August 7, 2024 11:11
@cpschau
Copy link
Contributor Author

cpschau commented Aug 7, 2024

Running workflow with 44 clusters yields following DH shares (pre- and post-update):

dh_shares_2020_pre dh_shares_2020_post
dh_shares_2030_pre dh_shares_2030_post
dh_shares_2045_pre dh_shares_2045_post

Copy link

github-actions bot commented Aug 7, 2024

Validator Report

I am the Validator. Download all artifacts here.
I'll be back and edit this comment for each new commit.

⚠️ Config changes detected!

Results may differ due to these changes:

diff --git a/config/config.yaml b/config/config.yaml
index 619f308..52aca79 100644
--- a/config/config.yaml
+++ b/config/config.yaml
@@ -4,7 +4,7 @@
 
 # docs in https://pypsa-eur.readthedocs.io/en/latest/configuration.html#run
 run:
-  prefix: 20240814limitseq
+  prefix: 20240814windcfs
   name:
   # - CurrentPolicies
   - KN2045_Bal_v4
@@ -105,12 +105,69 @@ atlite:
 renewable:
   onwind:
     cutout: europe-2019-sarah3-era5
+    resource:
+      smooth: false  #this is false until correction to onshore wind speeds from GWA implemented
+      #based on Vestas_V112_3MW, but changing hub_height from 80m with time
+      turbine:
+        2020:
+          V: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 25, 25]
+          POW: [0., 0., 0.005, 0.15, 0.3, 0.525, 0.905, 1.375, 1.95, 2.58, 2.96, 3.05, 3.06, 3.06, 0.]
+          hub_height: 80.
+          P: 3.06
+        2030:
+          V: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 25, 25]
+          POW: [0., 0., 0.005, 0.15, 0.3, 0.525, 0.905, 1.375, 1.95, 2.58, 2.96, 3.05, 3.06, 3.06, 0.]
+          hub_height: 90.
+          P: 3.06
+        2040:
+          V: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 25, 25]
+          POW: [0., 0., 0.005, 0.15, 0.3, 0.525, 0.905, 1.375, 1.95, 2.58, 2.96, 3.05, 3.06, 3.06, 0.]
+          hub_height: 100.
+          P: 3.06
   offwind-ac:
     capacity_per_sqkm: 6
     cutout: europe-2019-sarah3-era5
+    resource:
+      smooth: true
+      #based on NREL_ReferenceTurbine_2020ATB_5.5MW, but changing hub_height from 80m with time
+      turbine:
+        2020:
+          V: [3.0, 3.2, 3.5, 3.8, 4.0, 4.2, 4.5, 4.8, 5.0, 5.2, 5.5, 5.8, 6.0, 6.2, 6.5, 6.8, 7.0, 7.2, 7.5, 7.8, 8.0, 8.2, 8.5, 8.8, 9.0, 9.2, 9.5, 9.8, 10.0, 10.2, 10.5, 10.8, 11.0, 11.2, 11.5, 11.8, 12.0, 12.2, 12.5, 12.8, 13.0, 13.2, 13.5, 13.8, 14.0, 14.2, 14.5, 14.8, 15.0, 15.2, 15.5, 15.8, 16.0, 16.2, 16.5, 16.8, 17.0, 17.2, 17.5, 17.8, 18.0, 18.2, 18.5, 18.8, 19.0, 19.2, 19.5, 19.8, 20.0, 20.2, 20.5, 20.8, 21.0, 21.2, 21.5, 21.8, 22.0, 22.2, 22.5, 22.8, 23.0, 23.2, 23.5, 23.8, 24.0, 24.2, 24.5, 24.8, 25.0]
+          POW: [0.0, 0.127, 0.178, 0.237, 0.305, 0.381, 0.468, 0.564, 0.671, 0.789, 0.919, 1.061, 1.216, 1.385, 1.567, 1.765, 1.977, 2.205, 2.45, 2.711, 2.99, 3.287, 3.602, 3.937, 4.291, 4.665, 5.06, 5.477, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 0.0]
+          hub_height: 120.
+          P: 5.5
+        2030:
+          V: [3.0, 3.2, 3.5, 3.8, 4.0, 4.2, 4.5, 4.8, 5.0, 5.2, 5.5, 5.8, 6.0, 6.2, 6.5, 6.8, 7.0, 7.2, 7.5, 7.8, 8.0, 8.2, 8.5, 8.8, 9.0, 9.2, 9.5, 9.8, 10.0, 10.2, 10.5, 10.8, 11.0, 11.2, 11.5, 11.8, 12.0, 12.2, 12.5, 12.8, 13.0, 13.2, 13.5, 13.8, 14.0, 14.2, 14.5, 14.8, 15.0, 15.2, 15.5, 15.8, 16.0, 16.2, 16.5, 16.8, 17.0, 17.2, 17.5, 17.8, 18.0, 18.2, 18.5, 18.8, 19.0, 19.2, 19.5, 19.8, 20.0, 20.2, 20.5, 20.8, 21.0, 21.2, 21.5, 21.8, 22.0, 22.2, 22.5, 22.8, 23.0, 23.2, 23.5, 23.8, 24.0, 24.2, 24.5, 24.8, 25.0]
+          POW: [0.0, 0.127, 0.178, 0.237, 0.305, 0.381, 0.468, 0.564, 0.671, 0.789, 0.919, 1.061, 1.216, 1.385, 1.567, 1.765, 1.977, 2.205, 2.45, 2.711, 2.99, 3.287, 3.602, 3.937, 4.291, 4.665, 5.06, 5.477, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 0.0]
+          hub_height: 130.
+          P: 5.5
+        2040:
+          V: [3.0, 3.2, 3.5, 3.8, 4.0, 4.2, 4.5, 4.8, 5.0, 5.2, 5.5, 5.8, 6.0, 6.2, 6.5, 6.8, 7.0, 7.2, 7.5, 7.8, 8.0, 8.2, 8.5, 8.8, 9.0, 9.2, 9.5, 9.8, 10.0, 10.2, 10.5, 10.8, 11.0, 11.2, 11.5, 11.8, 12.0, 12.2, 12.5, 12.8, 13.0, 13.2, 13.5, 13.8, 14.0, 14.2, 14.5, 14.8, 15.0, 15.2, 15.5, 15.8, 16.0, 16.2, 16.5, 16.8, 17.0, 17.2, 17.5, 17.8, 18.0, 18.2, 18.5, 18.8, 19.0, 19.2, 19.5, 19.8, 20.0, 20.2, 20.5, 20.8, 21.0, 21.2, 21.5, 21.8, 22.0, 22.2, 22.5, 22.8, 23.0, 23.2, 23.5, 23.8, 24.0, 24.2, 24.5, 24.8, 25.0]
+          POW: [0.0, 0.127, 0.178, 0.237, 0.305, 0.381, 0.468, 0.564, 0.671, 0.789, 0.919, 1.061, 1.216, 1.385, 1.567, 1.765, 1.977, 2.205, 2.45, 2.711, 2.99, 3.287, 3.602, 3.937, 4.291, 4.665, 5.06, 5.477, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 0.0]
+          hub_height: 140.
+          P: 5.5
   offwind-dc:
     capacity_per_sqkm: 6
     cutout: europe-2019-sarah3-era5
+    resource:
+      smooth: true
+      #based on NREL_ReferenceTurbine_2020ATB_5.5MW, but changing hub_height from 80m with time
+      turbine:
+        2020:
+          V: [3.0, 3.2, 3.5, 3.8, 4.0, 4.2, 4.5, 4.8, 5.0, 5.2, 5.5, 5.8, 6.0, 6.2, 6.5, 6.8, 7.0, 7.2, 7.5, 7.8, 8.0, 8.2, 8.5, 8.8, 9.0, 9.2, 9.5, 9.8, 10.0, 10.2, 10.5, 10.8, 11.0, 11.2, 11.5, 11.8, 12.0, 12.2, 12.5, 12.8, 13.0, 13.2, 13.5, 13.8, 14.0, 14.2, 14.5, 14.8, 15.0, 15.2, 15.5, 15.8, 16.0, 16.2, 16.5, 16.8, 17.0, 17.2, 17.5, 17.8, 18.0, 18.2, 18.5, 18.8, 19.0, 19.2, 19.5, 19.8, 20.0, 20.2, 20.5, 20.8, 21.0, 21.2, 21.5, 21.8, 22.0, 22.2, 22.5, 22.8, 23.0, 23.2, 23.5, 23.8, 24.0, 24.2, 24.5, 24.8, 25.0]
+          POW: [0.0, 0.127, 0.178, 0.237, 0.305, 0.381, 0.468, 0.564, 0.671, 0.789, 0.919, 1.061, 1.216, 1.385, 1.567, 1.765, 1.977, 2.205, 2.45, 2.711, 2.99, 3.287, 3.602, 3.937, 4.291, 4.665, 5.06, 5.477, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 0.0]
+          hub_height: 120.
+          P: 5.5
+        2030:
+          V: [3.0, 3.2, 3.5, 3.8, 4.0, 4.2, 4.5, 4.8, 5.0, 5.2, 5.5, 5.8, 6.0, 6.2, 6.5, 6.8, 7.0, 7.2, 7.5, 7.8, 8.0, 8.2, 8.5, 8.8, 9.0, 9.2, 9.5, 9.8, 10.0, 10.2, 10.5, 10.8, 11.0, 11.2, 11.5, 11.8, 12.0, 12.2, 12.5, 12.8, 13.0, 13.2, 13.5, 13.8, 14.0, 14.2, 14.5, 14.8, 15.0, 15.2, 15.5, 15.8, 16.0, 16.2, 16.5, 16.8, 17.0, 17.2, 17.5, 17.8, 18.0, 18.2, 18.5, 18.8, 19.0, 19.2, 19.5, 19.8, 20.0, 20.2, 20.5, 20.8, 21.0, 21.2, 21.5, 21.8, 22.0, 22.2, 22.5, 22.8, 23.0, 23.2, 23.5, 23.8, 24.0, 24.2, 24.5, 24.8, 25.0]
+          POW: [0.0, 0.127, 0.178, 0.237, 0.305, 0.381, 0.468, 0.564, 0.671, 0.789, 0.919, 1.061, 1.216, 1.385, 1.567, 1.765, 1.977, 2.205, 2.45, 2.711, 2.99, 3.287, 3.602, 3.937, 4.291, 4.665, 5.06, 5.477, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 0.0]
+          hub_height: 130.
+          P: 5.5
+        2040:
+          V: [3.0, 3.2, 3.5, 3.8, 4.0, 4.2, 4.5, 4.8, 5.0, 5.2, 5.5, 5.8, 6.0, 6.2, 6.5, 6.8, 7.0, 7.2, 7.5, 7.8, 8.0, 8.2, 8.5, 8.8, 9.0, 9.2, 9.5, 9.8, 10.0, 10.2, 10.5, 10.8, 11.0, 11.2, 11.5, 11.8, 12.0, 12.2, 12.5, 12.8, 13.0, 13.2, 13.5, 13.8, 14.0, 14.2, 14.5, 14.8, 15.0, 15.2, 15.5, 15.8, 16.0, 16.2, 16.5, 16.8, 17.0, 17.2, 17.5, 17.8, 18.0, 18.2, 18.5, 18.8, 19.0, 19.2, 19.5, 19.8, 20.0, 20.2, 20.5, 20.8, 21.0, 21.2, 21.5, 21.8, 22.0, 22.2, 22.5, 22.8, 23.0, 23.2, 23.5, 23.8, 24.0, 24.2, 24.5, 24.8, 25.0]
+          POW: [0.0, 0.127, 0.178, 0.237, 0.305, 0.381, 0.468, 0.564, 0.671, 0.789, 0.919, 1.061, 1.216, 1.385, 1.567, 1.765, 1.977, 2.205, 2.45, 2.711, 2.99, 3.287, 3.602, 3.937, 4.291, 4.665, 5.06, 5.477, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 5.5, 0.0]
+          hub_height: 140.
+          P: 5.5
   offwind-float:
     capacity_per_sqkm: 6
     cutout: europe-2019-sarah3-era5
@@ -203,8 +260,6 @@ first_technology_occurrence:
     H2 Electrolysis: 2025
     H2 pipeline retrofitted: 2025
 
-
-
 costs:
   horizon: "mean" # "optimist", "pessimist" or "mean"
 
@@ -376,15 +431,6 @@ solving:
             2035: 400
             2040: 400
             2045: 400
-      Store:
-        co2 sequestered:
-          DE:
-            2020: 0
-            2025: 0
-            2030: 10000
-            2035: 20000
-            2040: 50000
-            2045: 80000
     limits_capacity_min:
       Generator:
         onwind:
Result plots comparison
Main branch Feature branch
Image not found in results Image not found in results
Image not found in results Image not found in results
Image not found in results Image not found in results
Image not found in results Image not found in results
Image not found in results Image not found in results
Image not found in results Image not found in results
Result files comparison
Numeric Status NMAE MAPE
../KN2045_Bal_v4/csvs/price_statistics.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/cumulative_cost.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/weighted_prices.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/nodal_cfs.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/cfs.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/nodal_capacities.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/capacities.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/energy.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/curtailment.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/costs.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/nodal_supply_energy.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/supply_energy.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/supply.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/metrics.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/market_values.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/nodal_costs.csv ⚠️ Missing
../KN2045_Bal_v4/csvs/prices.csv ⚠️ Missing

MAPE: Mean Absolute Percentage Error
NMAE: Mean Absolute Error on Min-Max Normalized Data
Status Thresholds: NMAE > 0.05 and MAPE > 5%

Comparing modify_dh_systems (b8525b6) with main (33916de).
Branch is 4 commits ahead and 4 commits behind.
Last updated on 2024-08-16 00:37:48 CEST.

Copy link

@lisazeyen lisazeyen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great @cpschau, I mainly have some code suggestions! Can you double check that the overall district heating share for Germany matches reported data from other sources? Can you plot total heat demand in Germany split by urban central, urban decentral and individual heating before and after your PR?

workflow/scripts/build_egon_data.py Show resolved Hide resolved
workflow/scripts/build_egon_data.py Outdated Show resolved Hide resolved
workflow/Snakefile Outdated Show resolved Hide resolved
workflow/Snakefile Outdated Show resolved Hide resolved
workflow/scripts/modify_district_heat_share.py Outdated Show resolved Hide resolved
@cpschau
Copy link
Contributor Author

cpschau commented Aug 12, 2024

Great @cpschau, I mainly have some code suggestions! Can you double check that the overall district heating share for Germany matches reported data from other sources? Can you plot total heat demand in Germany split by urban central, urban decentral and individual heating before and after your PR?

heat_demand_evolution

@lisazeyen
Copy link

Great @cpschau, I mainly have some code suggestions! Can you double check that the overall district heating share for Germany matches reported data from other sources? Can you plot total heat demand in Germany split by urban central, urban decentral and individual heating before and after your PR?

heat_demand_evolution

So the central heating demand in Germany is much larger than previously in 2020? That's fine if it is fits better to other data sources...

#
# SPDX-License-Identifier: MIT
"""
Load and prepares the data of the eGo^N DemandRegio project about district

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eGo^N is written here differently than in the other script, check for the right spelling.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can add the link to the dataset like this
eGo^N DemandRegio project <https://opendata.ffe.de/project/demandregio/>.

@lisazeyen
Copy link

@cpschau looks really good, just two small comments and check if you can get the dictionary internal_id from the metadata. Then this is ready to be merged!

@lisazeyen
Copy link

@lindnemi from my side this PR is ready to be merged!

@lindnemi
Copy link
Collaborator

lindnemi commented Aug 14, 2024

@cpschau great work! please complete the open points in our new PR requirements, then we're good to go:

  • Workflow with target rule ariadne_all completes without errors
  • The logic of export_ariadne_variables has been adapted to the changes
  • One or several figures that validate the changes in the PR have been posted as a comment
  • A brief description of the changes has been added to Changelog.md
  • The latest main has been merged into the PR
  • The config has a new prefix of the format YYYYMMDDdescriptive_title

thanks also to @lisazeyen for the detailed review

@lindnemi
Copy link
Collaborator

very clean and well documented PR :) 👍

@lindnemi lindnemi merged commit 0bd5246 into main Aug 14, 2024
0 of 2 checks passed
toniseibold pushed a commit that referenced this pull request Sep 23, 2024
* added new script for DH modifications

* function for update of urban heat loads

* Preserve aggregate DH demand

* remove subnodes

* extra rule for egon data retrieval

* Merge main

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add dh progress

* adjusted demand preservation for grid losses, improved comments

* change script structure; modify dh shares in csv before prepare_sector_network

* added docstring for script; changed logger config; excluded snakemake input from function; renamed inputs

* change snakemake input file name in Snakefile

* use technology mapping from metadata

* changed spelling of eGo^n and adapted reference of data set in docstring

* extended docstrings; added translation of 'Fernwaerme'; load snakemake input outside function

* extended changelog; added config prefix

* changed date format of prefix

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Micha <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants