Skip to content

Standardization fails if default value is not defined for a non-nullable array #86

@yruslan

Description

@yruslan

Describe the bug

The error is:

Caused by: java.lang.IllegalStateException: No default value defined for data type array
	at za.co.absa.standardization.types.CommonTypeDefaults.getDataTypeDefaultValue(CommonTypeDefaults.scala:63) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
	at za.co.absa.standardization.types.CommonTypeDefaults.$anonfun$getDataTypeDefaultValueWithNull$1(CommonTypeDefaults.scala:71) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
	at scala.util.Try$.apply(Try.scala:213) ~[scala-library-2.12.18.jar:?]
	at za.co.absa.standardization.types.CommonTypeDefaults.getDataTypeDefaultValueWithNull(CommonTypeDefaults.scala:71) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]

But the natural default value for an array is the empty array.

To Reproduce

  1. Create a schema with a non-nullable array column.
  2. Try to standardize data with this schema while not specifying a default value for the array column.
  3. See the error.

Expected behavior

Standardization should use empty array as the default value for array type automatically.

Business Value

--

Screenshots

--

Additional context

--

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions