Avoid anyOf, allOf, oneOf in schema structure in order to get better validation exception descriptions

[e-lo]

As somebody who has banged my head against many walls dealing with json-schema writing and validating json data to json-schemas, I can tell you that most validation tools for json-schema do not do a great job of telling you why a anyOf, allOf, oneOf statement fails - and can in fact provide misleading exception documentation.

It's unlikely that you can eliminate all of these...but anywhere they can be avoided you will be preventing headaches.

[vcschapp]

Thanks for this @e-lo ! We're unfortunately learning both how lacking the validation output can be with these constructs and, in parallel, how obfuscated the automatically-generated documentation can be.

We're talking internally about different solutions. There's a possibility (more than zero, but less than a probability) that we would switch to a different modeling meta-language if we feel the benefits outweigh the cost, including on the side of automated validation and/or documentation.

One thing I'm curious about is if you know any JSON Schema tricks that would allow us to express the same constructs but avoiding the anyOf/allOf/oneOf headaches. i.e. Can we write the same thing differently, or would we have to change the schema content itself?

[msbarry]

+1 to this. I just wrote a planetiler profile to parse and create a map from the first Overture data dump and I wanted to generate java code to represent the elements from the schema, but the json schema definitions were not sufficient for that. anyOf/allOf/oneOf confused jsonschema2pojo plus the field names don't actually match what shows up in the schema, mostly due to case mismatches and road metadata getting passed through as a raw json blob. So I had to write them by hand: https://github.com/onthegomap/planetiler/blob/overture/planetiler-core/src/main/java/com/onthegomap/planetiler/overture/OvertureSchema.java. This worked for a demo, but ideally one would be able to generate code in any language to parse the data directly from the schema.

[vcschapp]

One of our reasons for selecting JSON Schema was the existing open source toolchains, so if the way we're modeling is preventing those toolchains from working, I do think we need to rethink what we're doing, although developer bandwidth is a big limitation here...

I get the "field names don't usually" match issue as that seems to follow pretty much from the definition of those anyOf type constructs. But can you share in what way jsonschema2pojo became confused? I'd be interested in any error messages, problems, workarounds and kludges you needed to get the plane off the ground!

Please feel free to provide a painful level of detail if you're comfortable doing so - the more concrete data we have, the better.

[msbarry]

The issue with jsonschema2pojo is it just ignores allOf/anyOf completely (they have a proposal but it's not implemented because of the complexity in mapping those constraints to typed interfaces and classes). To reproduce:

• brew install jsonschema2pojo
• clone this repo
• run jsonschema2pojo -T YAMLSCHEMA -fe yaml -s schema/transportation/segment.yaml -t pojos/

pojos/Segment.java should have all of the fields in segment, but the generated Properties.java only has subType and connectors.

Even if jsonschema2pojo could generate typed classes from the overture schema, we wouldn't actually be able to use them to parse the data because:

• the schema uses fields names like subType but the overture data field name is subtype
• the schema nests the properties under a properties field but they are all top-level fields in the overture schema
• the schema describes how the transportation segment road attribute is structured, but in the actual data it's just a string

I understand that json schema lets you express more complex semantics but at the end of the day I'm just trying to generate a typed struct with optional fields that can be used to parse the syntax of an overture dump, more like how protobuf works.