Skip to content

Commit 5adef5b

Browse files
committed
docs: state that example key order is not significant
Two implementations asked independently whether the four languages should agree on the key order within an example object. They currently do not: Java emits Examples column order, Go sorts alphabetically because encoding/json sorts map keys when marshalling, and neither is wrong. A JSON object is unordered, so the answer is that order carries no meaning and implementations need not agree. Saying so explicitly is worth a sentence, because the alternative is three more implementations each deciding privately and someone eventually byte-comparing two reports and concluding they disagree when they do not. If byte-level reproducibility is ever required -- an attestation over the document is the obvious case -- that is a canonicalisation problem for the whole report, not something to solve inside one field. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
1 parent ad22c19 commit 5adef5b

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

specification/assets/provider-tck/report/conformance-report.schema.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,9 @@
152152
},
153153
"name": { "type": "string", "minLength": 1 },
154154
"example": {
155-
"description": "The Examples row this entry came from, as the row's parameters keyed by column header. Required for a scenario originating from a Scenario Outline, omitted otherwise.\n\nWithout it a report is ambiguous exactly where it matters most. Every row of an outline shares one scenario name, so eleven rows of a type-mismatch matrix produce eleven entries with identical feature and name; if one fails and ten pass, the report cannot say which failed, and a consumer keying on feature and name keeps whichever row it saw last.\n\nIt is a field rather than a naming convention because the parameters are the identity, and they come from the feature file rather than from any runner. Mandating a mangled name instead would put a separator, an ordering and an escaping rule into normative text that every implementation must reproduce byte-for-byte, and drift there is invisible until two reports silently fail to line up. Implementations were observed diverging on precisely this point before the field existed: one emitted the bare scenario name for all eleven rows, another appended its runner's example id, a third its runner's expanded title.\n\nValues are the cell contents verbatim, as strings, because Gherkin has no types -- no coercion, no trimming beyond what the parser does, so the report says what the table said.",
155+
"description": "The Examples row this entry came from, as the row's parameters keyed by column header. Required for a scenario originating from a Scenario Outline, omitted otherwise.\n\nWithout it a report is ambiguous exactly where it matters most. Every row of an outline shares one scenario name, so eleven rows of a type-mismatch matrix produce eleven entries with identical feature and name; if one fails and ten pass, the report cannot say which failed, and a consumer keying on feature and name keeps whichever row it saw last.\n\nIt is a field rather than a naming convention because the parameters are the identity, and they come from the feature file rather than from any runner. Mandating a mangled name instead would put a separator, an ordering and an escaping rule into normative text that every implementation must reproduce byte-for-byte, and drift there is invisible until two reports silently fail to line up. Implementations were observed diverging on precisely this point before the field existed: one emitted the bare scenario name for all eleven rows, another appended its runner's example id, a third its runner's expanded title.\n\nValues are the cell contents verbatim, as strings, because Gherkin has no types -- no coercion, no trimming beyond what the parser does, so the report says what the table said.
156+
157+
Key order is not significant and implementations are not required to agree on it: a JSON object is unordered, and implementations observed so far differ (one emits Examples column order, another sorts alphabetically because its language sorts map keys when marshalling). Compare these by parsing, not by comparing bytes. If byte-level reproducibility is ever needed -- for an attestation over the document, say -- that is a canonicalisation concern for the whole report rather than something to solve in this one field.",
156158
"type": "object",
157159
"minProperties": 1,
158160
"additionalProperties": { "type": "string" }

0 commit comments

Comments
 (0)