Compliant error reporting

[springcomp]

While working on this project I noticed that compliance tests that check for errors do not in fact enforce the specific error type. They only report success if any error occurred.

This issue is a reminder to please, consider reporting compliant errors and enforcing checks for speciied error types, like invalid-type, syntax, etc.

[eddycharly]

Not sure i fully understand the issue. Can you elaborate on what needs to be done ?

[springcomp]

Currently, the compliance test suite will mandate specific error types, like invalid-type or invalid-arity.
A standards compliant implementation must raise those compliant error types, although case and hyphens are loosely checked.

So an implementation that just checks that indeed, an error occured is somewhat OK. As is done in this repo.
But better is to check that the expected error is reported.

So where invalid-type is expected, a standards compliant implementation may raise any of the following errors:

• invalid type!
• an invalid type error occurred
• Invalid type: the function x expected argument was string but number was received instead.
• …

[springcomp]

jmespath/go-jmespath#83