v0.6.3

Fixes build problems resulting in empty jars in the previous v0.6.2 version.

v0.6.2

Adds support for Spark 3.4.x and 3.5.x
Note: Unfortunately this release didn't go well and artefacts produced in central repository are empty.

Release v0.6.1

• #97: Added Scala 2.13 support

Release v0.6.0

• #83: ErrorHandler trait and its several implementations, to offer error reporting between library and application that emerged during Spark data processing
• #82: OncePerSparkSession is now instantiatable even without providing Sparks session

Bugfix

• #95: Function isOfType might incorrectly reply on nonexistent column

Release 0.5.0

• #76: Added spark.sql.shuffle.partitions setting to the DefaultSparkConfiguration
• #78: Introduced ErrorMessage type originally from spark-data-standardization
• #80: Added enforceTypeOnNullTypeFields function to DataFrameImplicits casting all NullType fields to a target type

Release 0.4.1

• #74: Spark 3.3 added to project: build in projectmatrix for SBT, info added to README.md.

Release 0.4.0

• #65: Added support for generation of code coverage report using Jacoco
• #67: Added a class to automatically execute user defined code only once per SparkSession; most useful for UDF registration
• #69: Function transforming a column name to Spark expression evaluating the column
• #71: Fixed failing ScalaDoc class references and adding the code doc generation & its check to CI/CD

Release 0.3.2

• #61 and #62: Implemented functions stripParentPath and splitPath
• #57: Correcting Maven Badge
• #59: Correcting URL of the published homepage for the project

Release 0.3.1

• #51: Spark-commons depends on Spark-commons-test only in tests

Release 0.3.0

• #46: No more build warnings, build fails on any warning
• #7: Compile-time import of function transform; as a result, there are now specific modules for different Spark versions
  • spark-commons-spark2.4_2.11 - for Spark 2.4 and Scala 2.11
  • spark-commons-spark2.4_2.12 - for Spark 2.4 and Scala 2.12
  • spark-commons-spark3.2_2.12 - for Spark 3.2 and Scala 2.12
• #43: Implicit classes made more efficient by extending AnyVal where possible
• #38: Added the ability to load schema and DataFrame with schema from JSON
• #30: DataFrame.cacheIfNot implicit method added
• #29: Some protected properties of SparkTestConfig were made public for better configurability