Skip to content

Renewables (WP12)

Jump to bottom
Petrina edited this page Sep 16, 2024 · 19 revisions

Which applications/models are planned to get into operation in Phase 2

information for each application A,B,...

  • satellite nowcasting "DL-SATNOW" (GeoSphere algorithm, contactperson P. Gfäller): using LSA SAF DSSF_TOT near-realtime product+static fields and feeds it into IrradPhyDNet v1 (deep learning model) and produces nowcasts for the next 3 hours with 15-min timesteps, planned to be operational in 11/2024 as daytime/7 product
  • NWP2windpower (GeoSphere algorithm, contactperson I. Schicker): uses hectometric HARMONIE/AROME data and/or ECMWF-IFS (or ensemble if needed) and converts meteo forecast to power production forecast using parametric powercurves and a small set of metadata (location, hub height, rotor diameter, rated power). Planned to be operational 11/2024 as on-demand post-processing product
  • PV site forecasting (GeoSphere algorithm, contactperson P. Papazek): uses analysis and observation of PV and close met. sites of various parameters related to PV production in addition to parameters from NWP or DL-SATNOW depending on forecast range. Location optimized PV forecasts are generated by site optimized models. Planned to be operational 11/2024 as on-demand post-processing product and higher resolved daytime/7 product.
  • Wind farm analogs (DHMZ algorithm, contactperson I. Vujec): uses ECMWF-IFS and ERA-5, generates training dataset for specific event and location, and generates analog-based forecast of wind speed and power production (deterministic or probabilistic). Planned to be operational 11/2025 (or possibly 02/2025) as an on-demand post-processing product.
  • application A: doing this and that.., planned to be operational in MM/YYYY
  • ...

Status: ready to go, under development

  • satellite nowcasting "DL-SATNOW":
    • development finished and ready to be incorporated
    • yaml/python environment, currently porting to singularity (ongoing)
    • currently regular run, no trigger needed for pilot regions, will be needed for end of phase II for pan-European application
    • fixed for Denmark/Austria but being ported to pan-European
  • NWP2windpower:
    • development finished, might need one round of "does it really run smoothly in the python environment with all models"
    • yaml/python environment, being ported to singularity
    • should run post-triggered NWP for storms for pilot regions (Belgian offshore, Austria) but could run regularly using either ECMWF-IFS or global DT (if we really want that)
    • fixed domains based on available free wind farm metadata (location, hub heights, rotor diameter, rated power, etc.)
  • PV site forecasting:
    • Basic version finished, optimization, particularly of new sites under development, refined version including synthetic data generation under development
    • yaml/python environment
    • currently regular run, no trigger needed
    • set-up for test-sites Denmark/Austria to be extended on further sites
  • Wind farm analogs:
    • Basic version finished, optimization, particularly using ECMWF-IFS (currently analyzed on the ALADIN NWP) is under development
    • Python environment
    • Currently regular run, no trigger needed
    • Currently tested for test-sites in Croatia, can be extended on further sites
  • application A:
    • development finished? ready to be incorporated? still under development?
    • ecflow exists already? other scripting environment? other workflow environment?
    • trigger needed from NWP vs. regular runs? .. give any useful information
    • NRT testing plans? when? fixed domains? flexible domains? ...
  • application B: ...

Triggering

  • satellite nowcasting "DL-SATNOW":
    • needed from 2025 onwards, triggered on day of predicted convective cases and should then run every 15 minutes until end of detected case for non-pilot region cases. Could be coordinated with triggering of hectometric run of the day of event. Threshold-based, index-based
    • CAPE, precipitation, showalter(? 'the end my friend'-index),domain center (latlon bounds will be calculated according to domain center)
    • pilot region (Denmark, Austria) cases planned to run daytime/7, pan-European implementation under development
  • NWP2windpower:
    • triggered with NWP runs for storm events (or later defined wind energy events), threshold-based currently
    • wind speed, right now 10m but ideally 100m, gustiness
    • pilot region, in later stage where we have wind farm metadata
  • PV site forecasting:
    • No classical triggering, runs post DL-SATNOW / NWP if triggered in order to provide site tailored PV estimations
    • pilot sites only
  • Wind farm analogs:
    • No classical triggering, runs post NWP if triggered in order to provide site tailored wind speed and power production estimations
    • Pilot sites only
  • application A:
    • approach: threshhold-based (optihthred...) or other? manual?
    • data/parameters needed for triggering
    • pilot regions?
  • application B: ...

Targeted platform (HPC needed?, EWC?)

  • satellite nowcasting "DL-SATNOW":
    • inference mode: needs CPU so probably EWC (or even git could work but not tested, multi-core beneficial to pre-processing time), training: needs GPU (LUMI/Leonardo)
    • inference: testing on git CI/CD September onwards, training: not yet tested on GPU
    • inference: linux env with ~8 or 16 GB RAM works, runtime incl. data loading, pre-processing and plot generation currently 1 - 2 minutes max.; training: GPU platform (at least 2 would be good), runtime ~ 4 days for domain covering size of Austria
  • NWP2windpower:
    • needs CPU, very leightweight model
    • LUMI/Leonardo would be overkill.
    • ~ 8GB RAM would suffice, runtime depends a bit what else is going on on the resp. machine
  • PV site forecasting:
    • needs CPU – GPU optionally for training – leightweight/flat ANN model
    • LUMI/Leonardo possible.
    • ~ 8GB RAM would suffice for few sites, per-site: training / preprocessing < 10 minutes, prediction with trained model (operational cases) fast – data-retrieval is a potential bottleneck.
  • Wind farm analogs:
    • Needs CPU.
    • LUMI/Leonardo would be overkill.
    • 8GB RAM would suffice for a few sites, data-retrieval is a potential bottleneck, easily parallelizable.
  • application A:
    • needs HPC? target is LUMI/Leonardo?
    • already tested there?
    • expected resources needed? expected runtime? etc.
  • application B: ..

Input needs (NWP, other data/auxiliary data), where to access

  • satellite nowcasting "DL-SATNOW":
    • LSA SAF DSSF_TOT near realtime, netcdf
    • topographic data, ideally netcdf but tif should also work
    • LSA SAF near realtime archive
    • no inline/direct access needed for now
  • NWP2windpower:
    • yes, grib 2 is fine but nasty
    • netcdf or grib
    • FDB or HPC inline --> needs discussion on how to best connect with NWP output for time-saving
    • inline could speed up
    • probably z0 forecast and additional heights above ground, ideally 10-minute frequency or 5-minute to account for traders needs
    • not for this code but for next model iteration to more advanced ML-model historic data is needed
  • PV site forecasting:
    • NWP, DL-SATNOW as netcdf if used – code can run with observation/analysis only (csv) using CAMS radiation time series
    • CAMS radiation time-series in 15 minute updates, further atmospheric parameters from close met. site or analysis, PV observations including available meta-information (if shareable – otherwise pvlib standard modules will be used), site’s exact geo-location (preferably lat/lon, alternatively addresses will be geocoded)
    • Training of ML-model needs historic data of all inputs and PV (=target) – semi-synthetic data will be accessible for reduced data sources in the future
    • no inline/direct access needed currently
  • Wind farm analogs:
    • NWP data.
    • NWP and synthetic (ERA-5) historic data needed. Training of analog-based model needs historic data of all inputs and wind speed/power production – semi-synthetic data will be accessible for reduced data sources in the future.
    • Wind-farm information details needed.
  • application A:
    • NWP data needed? (format Grib edition 2 is fine?),
    • other data (types)?
    • to be accessed from FDB/...,
    • some inline/direct access e.g. on HPC needed? (-> potential speed up e.g. for AQ models)
    • any special needs? parameters not yet considered in NWP output, ...
    • historic data needed?
  • application B: ...

Expected output, we need to sort out the format management

  • satellite nowcasting "DL-SATNOW": netcdf, plots as png if wanted, archiving not necessarily needed for data older than 2 days. One forecast Austria <1MB for +3hours (=12 time steps) as compressed netcdf
  • NWP2windpower: csv files, plots as png if wanted, archiving not necessarily needed for data older than 2 days. Size depends on number of turbines/wind farms needed to be calculated and forecast time steps, number of models
  • PV site forecasting: csv files (will be very small for one site and a trigger day), archiving not necessarily needed. Size depends on number of sites needed to be calculated and forecast time steps / horizon (currently: few MB each)
  • Wind farm analogs: csv files (will be very small for one site and a trigger day), archiving not necessarily needed. Size depends on the number of sites needed to be calculated and forecast time steps / horizon (currently: few MB each)
  • application A: the application outputs GRIB 2, netcdf,..., plots, whatever..., data amount?, archiving needed?
  • application B: ...

NRT monitoring

  • satellite nowcasting "DL-SATNOW": real time monitoring via failure email
  • NWP2windpower: assessment after on-demand launch
  • PV site forecasting: assessment after on-demand launch
  • Wind farm analogs: assessment after on-demand launch
  • application A: what/how/who on real time monitoring, assessment during and after on-demand launch
  • application B: ...

What is the timeline? Milestones events?

  • satellite nowcasting "DL-SATNOW": 11/2024 first version promised to EMCWF, milestone pre-operational implementation 31/1/2025
  • NWP2windpower: 11/2024 first version promised to EMCWF, milestone pre-operational implementation 31/1/2025
  • PV site forecasting: 11/2024 first version promised to EMCWF, milestone pre-operational implementation 31/1/2025
  • Wind farm analogs: 02/2025 first version promised to ECMWF
  • application A: Milestone event according the technical proposal is DD/MM/YYYY, if not milestone .. what's the timeline?
  • application B: ...

PoC for sector-specific working groups

(needed to help to develop and build up the end-to-end workflow in operations)

  • satellite nowcasting "DL-SATNOW":
    • Pascal Gfäller (developer)
    • Irene Schicker (user/restarting)
  • NWP2windpower:
    • Irene Schicker (developer)
  • PV site forecasting:
    • Petrina Papazek (developer)
  • Wind farm analogs:
    • Ivan Vujec (developer)
  • application A:
    • Name A (developer)
    • Name B (developer)
    • Name C (user X)
    • ..
  • application B:

PoC for 3rd line support

(to be contacted by 2nd line in case of problems with the operational workflow for this application)

  • satellite nowcasting "DL-SATNOW":
    • Pascal Gfäller
    • Irene Schicker
  • NWP2windpower:
    • Irene Schicker
    • Petrina Papazek
  • PV site forecasting:
    • Petrina Papazek
    • Irene Schicker
  • Wind farm analogs:
    • Ivan Vujec
  • application A:
    • Name A
    • Name B
    • ...
  • application B: ...
Clone this wiki locally