Merge pull request #1034 from ie3-institute/rel/sp/#1033-release-5.0

Release version 5.0

Author: sebastian-peter

.gitignore

@@ -242,4 +242,9 @@ docs/readthedocs/_build
 **/*.synctex.gz
 **/*.synctex(busy)
 
-docs/javadoc
+### Metals ###
+.metals/
+.bloop/
+project/**/metals.sbt
+
+docs/javadoc

AUTHORS

@@ -1,10 +1,14 @@
-Copyright (c) 2023, Institute of Energy Systems, Energy Efficiency and Energy Economics (ie3)
+Copyright (c) 2024, Institute of Energy Systems, Energy Efficiency and Energy Economics (ie3)
 All rights reserved.
 
 Lead Developers:
     - Johannes Hiry
     - Chris Kittl
     - Debopama Sen-Sarma
+    - Thomas Oberließen
+    - Daniel Feismann
+    - Johannes Bao
+    - Sebastian Peter
 
 Main Contributers:
     - Johannes Hiry - https://github.com/johanneshiry
@@ -20,3 +24,8 @@ Main Contributers:
     - Marius Staudt - https://github.com/staudtMarius
     - Lara Roumeliotis - https://github.com/lararou
     - Vicky Bung - https://github.com/vickybung1
+    - Daniel Feismann - https://github.com/danielfeismann
+    - Johannes Bao - https://github.com/jo-bao
+    - Julian Hohmann - https://github.com/julianhohmann
+    - Simon Huette - https://github.com/SimonHuette
+    - Pierre Petersmeier - http://github.com/pierrepetersmeier

CHANGELOG.md

@@ -6,11 +6,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased/Snapshot]
 
+## [5.0.0] - 2024-03-06
+
 ### Added
+- Formatting Spotless Groovy import order [#960](https://github.com/ie3-institute/PowerSystemDataModel/issues/960)
+- Implementing missing typical methods in `Try` [#970](https://github.com/ie3-institute/PowerSystemDataModel/issues/970)
+- Added log warning when using `SwitchInputs` with `parallelDevices` parameter [#840](https://github.com/ie3-institute/PowerSystemDataModel/issues/840)
+- Validation for `EvcsInput` [#1000](https://github.com/ie3-institute/PowerSystemDataModel/issues/1000)
+- Scaling method in system participant copy builders [#1011](https://github.com/ie3-institute/PowerSystemDataModel/issues/1011)
+- Added separate field for maximum power limit for DC to evtype [#876](https://github.com/ie3-institute/PowerSystemDataModel/issues/876)
+- Added test for invalid input data in `CsvRawGridSource` [#1021](https://github.com/ie3-institute/PowerSystemDataModel/issues/1021)
+- Added `CsvThermalGridSource` [#1009](https://github.com/ie3-institute/PowerSystemDataModel/issues/1009)
+- Enhance documentation for CSV timeseries [#825](https://github.com/ie3-institute/PowerSystemDataModel/issues/825)
 
 ### Fixed
+- Fixed Couchbase integration tests that randomly failed [#755](https://github.com/ie3-institute/PowerSystemDataModel/issues/755)
+- Fixed hyperlink in line documentation [#965](https://github.com/ie3-institute/PowerSystemDataModel/issues/965)
+- Fixed some tests no failing when they should [#958](https://github.com/ie3-institute/PowerSystemDataModel/issues/958)
+- Fixed thermal-house-documentation [#873](https://github.com/ie3-institute/PowerSystemDataModel/issues/873)
+- Fixed ElectricVehicle Documentation [#875](https://github.com/ie3-institute/PowerSystemDataModel/issues/875)
+- Fixed Equal behavior of `ConnectorResult` [#1001](https://github.com/ie3-institute/PowerSystemDataModel/issues/1001) 
+- Fixed test for invalid input data in `CsvGraphicSource` [#1022](https://github.com/ie3-institute/PowerSystemDataModel/issues/1022)
 
 ### Changed
+- Changing from comparing strings to comparing uuids in `EntitySource.findFirstEntityByUuid` [#829](https://github.com/ie3-institute/PowerSystemDataModel/issues/829)
+- Adding JavaDoc to `EntitySource.safeMapGet` [#828](https://github.com/ie3-institute/PowerSystemDataModel/issues/828)
+- Abstracting some methods in `ValidationUtils` [#852](https://github.com/ie3-institute/PowerSystemDataModel/issues/852)
+- Changes to Energy Management inputs:
+  - `EmInput` should not be connected to the grid [#955](https://github.com/ie3-institute/PowerSystemDataModel/issues/955)
+  - System participants now reference the em entity [#957](https://github.com/ie3-institute/PowerSystemDataModel/issues/957)
+- Enhancing the error message for coordinate sources with invalid column names [#670](https://github.com/ie3-institute/PowerSystemDataModel/issues/670) 
+- Allowing for additional unused columns in sources [#839](https://github.com/ie3-institute/PowerSystemDataModel/issues/839)
+- Improving column name validation to only run once per source [#849](https://github.com/ie3-institute/PowerSystemDataModel/issues/849)
+- Refactored and abstracted `EntitySource`s and `EntityData` creation [#969](https://github.com/ie3-institute/PowerSystemDataModel/issues/969)
+- Updated contributing.md [#737](https://github.com/ie3-institute/PowerSystemDataModel/issues/737)
+- Don't throw exceptions for not yet implemented validations [#879](https://github.com/ie3-institute/PowerSystemDataModel/issues/879)
+- `CsvDataSource` throws exceptions on error [#954](https://github.com/ie3-institute/PowerSystemDataModel/issues/954)
+- Removing `uuid` as required column from input and result time series [#826](https://github.com/ie3-institute/PowerSystemDataModel/issues/826)
+- Removing the support for the old csv format that was marked `deprecated` back in version `1.1.0` [#795](https://github.com/ie3-institute/PowerSystemDataModel/issues/795)
+- BREAKING: Updating PowerSystemUtils dependency to 2.2 [#1006](https://github.com/ie3-institute/PowerSystemDataModel/issues/1006)
 
 ## [4.1.0] - 2023-11-02
 
@@ -34,6 +68,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Fixed
 - Fixed wrong rated power unit hint [#804](https://github.com/ie3-institute/PowerSystemDataModel/issues/804)
 - Fixed wrong hash code generation of ConnectorResult [#817](https://github.com/ie3-institute/PowerSystemDataModel/issues/817) 
+- Fixed wrong path for timeSeriesMapping [#886](https://github.com/ie3-institute/PowerSystemDataModel/issues/886)
 
 ### Changed
 - Removing deprecated classes and methods [#540](https://github.com/ie3-institute/PowerSystemDataModel/issues/540)
@@ -42,6 +77,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Use nio paths instead of strings for file path [#723](https://github.com/ie3-institute/PowerSystemDataModel/issues/723)
 - Data source will throw an exceptions instead of returning an empty optionals [#707](https://github.com/ie3-institute/PowerSystemDataModel/issues/707)
 - Improving `ValidationUtils` [#758](https://github.com/ie3-institute/PowerSystemDataModel/issues/758)
+- conversion from rst to markdown myst[#979](https://github.com/ie3-institute/PowerSystemDataModel/pull/985)
 
 ## [3.0.0] - 2023-02-16
 
@@ -236,7 +272,8 @@ coordinates or multiple exactly equal coordinates possible
 -   CsvDataSource now stops trying to get an operator for empty operator uuid field in entities
 -   CsvDataSource now parsing multiple geoJson strings correctly
 
-[Unreleased/Snapshot]: https://github.com/ie3-institute/powersystemdatamodel/compare/4.1.0...HEAD
+[Unreleased/Snapshot]: https://github.com/ie3-institute/powersystemdatamodel/compare/5.0.0...HEAD
+[4.1.0]: https://github.com/ie3-institute/powersystemdatamodel/compare/4.1.0...5.0.0
 [4.1.0]: https://github.com/ie3-institute/powersystemdatamodel/compare/4.0.0...4.1.0
 [4.0.0]: https://github.com/ie3-institute/powersystemdatamodel/compare/3.0.0...4.0.0
 [3.0.0]: https://github.com/ie3-institute/powersystemdatamodel/compare/2.1.0...3.0.0

CONTRIBUTING.md

@@ -16,7 +16,7 @@ With this document we would like to give you some orientation on how you can con
 ## Testing and reporting bugs
 We really appreciate your usage of this project.
 Whenever you find a bug, it would be nice to check, if this isn't a feature to us. :wink:
-You may find extensive information about the intended behaviour when you [Read the Docs](https://powersystemdatamodel.readthedocs.io/en/latest/) or within the [javadoc](https://ie3-institute.github.io/PowerSystemDataModel/javadoc/).
+You may find extensive information about the intended behaviour when you [Read the Docs](https://powersystemdatamodel.readthedocs.io/en/latest/).
 If you still think it's a bug, please raise an [issue](https://guides.github.com/features/issues/) for us.
 Considering the following aspects in your inquiry, assists us in helping you:
 
@@ -67,9 +67,6 @@ Before marking the pull request as 'ready to review', take these precautionary a
 `gradle finalizePR` summarizes all of these steps .
 
 ## For any doubts
-... please contact
-* Johannes (@johanneshiry),
-* Debopama (@sensarmad) or
-* Chris (@ckittl)
+... please contact [us](AUTHORS)
 
 We are happy to help! :smiley:

Jenkinsfile

@@ -451,7 +451,6 @@ def constantBranchesProps() {
     [
       string(defaultValue: '', description: '', name: 'deploy', trim: true)
     ]),
-    [$class: 'RebuildSettings', autoRebuild: false, rebuildDisabled: false],
     [$class: 'ThrottleJobProperty', categories: [], limitOneJobWithMatchingParams: false, maxConcurrentPerNode: 0, maxConcurrentTotal: 0, paramsToUseForLimit: '', throttleEnabled: true, throttleOption: 'project']
   ])
 }

build.gradle

@@ -4,9 +4,9 @@ plugins {
   id 'maven-publish'
   id 'signing'
   id 'pmd' // code check, working on source code
-  id 'com.diffplug.spotless' version '6.22.0' //code format
-  id 'com.github.spotbugs' version '5.2.1' // code check, working on byte code
-  id 'de.undercouch.download' version '5.5.0'
+  id 'com.diffplug.spotless' version '6.25.0' //code format
+  id 'com.github.spotbugs' version '6.0.8' // code check, working on byte code
+  id 'de.undercouch.download' version '5.6.0'
   id 'kr.motd.sphinx' version '2.10.1' // documentation generation
   id 'jacoco' // java code coverage plugin
   id "org.sonarqube" version "4.4.1.3373" // sonarqube
@@ -17,8 +17,8 @@ ext {
   //version (changing these should be considered thoroughly!)
   javaVersion = JavaVersion.VERSION_17
   groovyVersion = "4.0"
-  groovyBinaryVersion = "4.0.15"
-  testcontainersVersion = '1.19.1'
+  groovyBinaryVersion = "4.0.19"
+  testcontainersVersion = '1.19.7'
 
   scriptsLocation = 'gradle' + File.separator + 'scripts' + File.separator //location of script plugins
 }
@@ -53,7 +53,7 @@ repositories {
 dependencies {
 
   //	 ie³ power system utils
-  implementation 'com.github.ie3-institute:PowerSystemUtils:2.0'
+  implementation 'com.github.ie3-institute:PowerSystemUtils:2.2.1'
 
   implementation 'tech.units:indriya:2.2'
 
@@ -70,10 +70,10 @@ dependencies {
   // testing
   testImplementation "org.apache.groovy:groovy:$groovyBinaryVersion"
 
-  testImplementation 'org.junit.jupiter:junit-jupiter:5.10.0'
+  testImplementation 'org.junit.jupiter:junit-jupiter:5.10.2'
   testImplementation "org.spockframework:spock-core:2.3-groovy-$groovyVersion"
   testImplementation 'org.objenesis:objenesis:3.3' // Mock creation with constructor parameters
-  testImplementation 'net.bytebuddy:byte-buddy:1.14.9' // Mocks of classes
+  testImplementation 'net.bytebuddy:byte-buddy:1.14.12' // Mocks of classes
 
   // testcontainers (docker framework for testing)
   testImplementation "org.testcontainers:testcontainers:$testcontainersVersion"
@@ -83,19 +83,19 @@ dependencies {
   testImplementation "org.testcontainers:couchbase:$testcontainersVersion"
 
   // logging
-  implementation platform('org.apache.logging.log4j:log4j-bom:2.21.1')
+  implementation platform('org.apache.logging.log4j:log4j-bom:2.23.0')
   implementation 'org.apache.logging.log4j:log4j-api' // log4j
   implementation 'org.apache.logging.log4j:log4j-core' // log4j
   implementation 'org.apache.logging.log4j:log4j-slf4j-impl' // log4j -> slf4j
 
   // Databases
-  implementation 'org.influxdb:influxdb-java:2.23'
-  implementation 'com.couchbase.client:java-client:3.4.11'
-  runtimeOnly 'org.postgresql:postgresql:42.6.0' // postgresql jdbc driver required during runtime
+  implementation 'org.influxdb:influxdb-java:2.24'
+  implementation 'com.couchbase.client:java-client:3.5.3'
+  runtimeOnly 'org.postgresql:postgresql:42.7.2' // postgresql jdbc driver required during runtime
 
-  implementation 'commons-io:commons-io:2.15.0' // I/O functionalities
-  implementation 'org.apache.commons:commons-compress:1.24.0' // I/O functionalities
-  implementation 'org.apache.commons:commons-lang3:3.13.0'
+  implementation 'commons-io:commons-io:2.15.1' // I/O functionalities
+  implementation 'commons-codec:commons-codec:1.16.1' // needed by commons-compress
+  implementation 'org.apache.commons:commons-compress:1.26.0' // I/O functionalities
 }
 
 tasks.withType(JavaCompile) {

docs/readthedocs/gettingstarted.md

@@ -0,0 +1,75 @@
+# Getting started
+Welcome, dear fellow of bottom up power system modelling!
+This section is meant to give you some help getting hands on our project.
+If you feel, something is missing, please contact us!
+
+## Requirements
+
+Java > v 17
+
+## Where to get
+
+Checkout latest from [GitHub](https://github.com/ie3-institute/PowerSystemDataModel) or use maven for dependency
+management:
+
+### Stable releases
+
+On [Maven central](https://search.maven.org/artifact/com.github.ie3-institute/PowerSystemDataModel):
+
+```xml
+<dependency>
+  <groupId>com.github.ie3-institute</groupId>
+  <artifactId>PowerSystemDataModel</artifactId>
+  <version>2.1.0</version>
+</dependency>
+```
+
+### Snapshot releases
+
+Available on [OSS Sonatype](https://s01.oss.sonatype.org/).
+Add the correct repository:
+
+```xml
+<repositories>
+  <repository>https://s01.oss.sonatype.org/content/repositories/snapshots</repository>
+</repositories>
+```
+
+and add the dependency:
+
+```xml
+<dependency>
+  <groupId>com.github.ie3-institute</groupId>
+  <artifactId>PowerSystemDataModel</artifactId>
+  <version>3.0-SNAPSHOT</version>
+</dependency>
+```
+
+## Important changes
+
+With the release of `PowerSystemDataModel` version `5.0` the support for the old csv file format will be fully removed.
+It was already marked as `deprecated` back in version `1.1.0`. For those who are still using a model in the old csv format
+the following guide will provide a fast and easy way to convert old format into the new one.
+
+- Since the support is removed in version `5.0`, the `PowerSystemDataModel` version `3.x` or `4.x` must be 
+  used to read the old format. The `PSDM` will automatically write the output model in the new csv format.
+
+
+``` java
+    /* Parameterization */
+    String gridName = "gridWithOldFormat";
+    String csvSep = ",";
+    Path folderPath = Path.of("PATH_TO_THE_FOLDER");
+    boolean isHierarchic = false;
+    Path output = Path.of("PATH_OF_THE_OUTPUT_FOLDER");
+    FileNamingStrategy namingStrategy = new FileNamingStrategy();
+    
+    /* Reading the old format */
+    JointGridContainer container = CsvJointGridContainerSource.read(gridName, csvSep, folderPath, isHierarchic);
+    
+    /* Writing in the new format */
+    CsvFileSink sink = new CsvFileSink(output, namingStrategy, csvSep);
+    sink.persistJointGrid(container);
+```
+
+

docs/readthedocs/gettingstarted.rst

@@ -0,0 +1,34 @@
+# Documentation of the PowerSystemDataModel
+
+Welcome to the documentation of the PowerSystemDataModel.
+It provides an extensive data model capable of modelling energy systems with high granularity e.g. for bottom-up simulations.
+Additionally, useful functions to process, augment and furnish model i/o information is provided.
+Effective handling of geographic information related to power grids is also possible.
+
+```{toctree}
+:caption: 'Contents:'
+:maxdepth: 2
+
+gettingstarted
+models/models
+io/basiciousage
+```
+
+## Contact the (Main) Maintainers
+
+If you feel, something this missing, wrong or misleading, please contact one of our main contributors:
+
+> - [@sensarmad](https://github.com/sensarmad)
+> - [@t-ober](https://github.com/t-ober)
+> - [@danielfeismann](https://github.com/danielfeismann)
+> - [@sebastian-peter](https://github.com/sebastian-peter)
+> - [@jo-bao](https://github.com/jo-bao)
+> - [@julianhohmann](https://github.com/julianhohmann)
+
+Hat tip to all other contributors!
+
+# Indices and tables
+
+- {ref}`genindex`
+- {ref}`modindex`
+- {ref}`search`