Skip to content

Ensure deterministic ordering for Dictionary keys #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rickhohler wants to merge 5 commits into from
Closed

Ensure deterministic ordering for Dictionary keys #22

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8cb1f83
fix: Ensure deterministic key ordering
rickhohler Feb 5, 2026
8e70638
swift format -i -r .
mattt Feb 6, 2026
eb8aaa0
Delete SortingTests.swift
mattt Feb 6, 2026
7c9543c
Use reflection to deterministically order dictionary keys
mattt Feb 6, 2026
7df821b
Remove unused Value.from constructor
mattt Feb 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,10 @@ and implements the following features:
- [x] Configurable flatten depth to limit the depth of key folding
- [x] Collision avoidance so folded keys never collide with existing sibling keys

> [!NOTE]
> `TOONEncoder` preserves the field order defined by `Encodable` types (the order keys are encoded).
> When encoding Swift `Dictionary` values, keys are sorted to ensure a deterministic output.

### TOONDecoder

`TOONDecoder` conforms to **TOON specification version 3.0** (2025-11-24)
Expand Down
8 changes: 6 additions & 2 deletions Sources/ToonFormat/Encoder.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -810,6 +810,8 @@ extension TOONEncoder {

private var container: [String: Value] = [:]
private var keyOrder: [String] = [] // Track insertion order
// Swift Dictionary encoding uses an internal DictionaryCodingKey; detect it to sort keys deterministically.
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The detection of Dictionary encoding via String reflection is fragile and relies on internal Swift implementation details. The string "DictionaryCodingKey" is an internal Swift type name that could change in future Swift versions, breaking this functionality silently.

Consider these alternatives:

  1. Document this as a known limitation that depends on Swift internals
  2. Add a test that explicitly verifies the reflection string contains "DictionaryCodingKey" when encoding a Dictionary, so that any Swift version upgrade that breaks this will be caught
  3. Explore whether there's a more robust way to differentiate Dictionary encoding from struct encoding (though this may not be possible with the Codable API)

Given that this is a critical feature for deterministic output, it would be prudent to have explicit test coverage that verifies the detection mechanism is working as expected.

Suggested change
// Swift Dictionary encoding uses an internal DictionaryCodingKey; detect it to sort keys deterministically.
/// Heuristic: Swift's `Dictionary` encoding uses an internal `DictionaryCodingKey` type.
///
/// We detect this by checking whether `String(reflecting: Key.self)` contains
/// the substring `"DictionaryCodingKey"`, and if so, we treat the container
/// as a dictionary and sort its keys deterministically.
///
/// This relies on Swift's internal implementation details and is therefore
/// inherently fragile: if the type name changes in a future Swift version,
/// this detection will stop working and dictionary key ordering may become
/// non-deterministic again without a compile-time error.
///
/// This limitation should be documented and covered by tests that assert
/// the presence of `"DictionaryCodingKey"` when encoding a `Dictionary`,
/// so that Swift upgrades which change the internal type name are caught.

Copilot uses AI. Check for mistakes.
private let isDictionaryCodingKey = String(reflecting: Key.self).contains("DictionaryCodingKey")
Comment on lines +813 to +814
Copy link
Copy Markdown
Collaborator

@mattt mattt Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind of hacky, but it lets us have nice behavior where Dictionary keys are sorted but struct fields retain ordering.


init(encoder: Encoder, codingPath: [CodingKey]) {
self.encoder = encoder
Expand Down Expand Up @@ -1095,12 +1097,14 @@ extension TOONEncoder {
}

func finishEncoding() {
encoder.storage.append(.object(container, keyOrder: keyOrder))
let finalKeyOrder = isDictionaryCodingKey ? container.keys.sorted() : keyOrder
encoder.storage.append(.object(container, keyOrder: finalKeyOrder))
}

deinit {
// Ensure the container is finished when it goes out of scope
encoder.storage.append(.object(container, keyOrder: keyOrder))
let finalKeyOrder = isDictionaryCodingKey ? container.keys.sorted() : keyOrder
encoder.storage.append(.object(container, keyOrder: finalKeyOrder))
Comment on lines +1100 to +1107
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same logic for determining finalKeyOrder is duplicated in both finishEncoding and deinit. Consider extracting this into a computed property or private method to reduce duplication and improve maintainability. For example:

private var finalKeyOrder: [String] {
    isDictionaryCodingKey ? container.keys.sorted() : keyOrder
}

This would make both methods simpler and ensure the logic stays consistent if it needs to be updated in the future.

Copilot uses AI. Check for mistakes.
}
}
}
Expand Down
12 changes: 6 additions & 6 deletions Sources/ToonFormat/Value.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,14 +182,14 @@ enum Value: Equatable {

if let dictionaryValue = value as? [String: Any] {
var object: [String: Value] = [:]
var keyOrder: [String] = []
for (key, value) in dictionaryValue {
if !keyOrder.contains(key) {
keyOrder.append(key)
}
// Sort keys to ensure deterministic order
let sortedKeys = dictionaryValue.keys.sorted()

for key in sortedKeys {
let value = dictionaryValue[key]!
object[key] = Value.from(value)
}
return .object(object, keyOrder: keyOrder)
return .object(object, keyOrder: sortedKeys)
}

return .null
Expand Down
51 changes: 51 additions & 0 deletions Tests/ToonFormatTests/EncoderTests.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,57 @@ struct EncoderTests {
#expect(quoteKeyResult.contains("\"he said \\\"hi\\\"\": 1"))
}

@Test func dictionaryKeyOrderingIsDeterministic() async throws {
let orderedPairs: [(String, Int)] = [
("alpha", 1),
("bravo", 2),
("charlie", 3),
("delta", 4),
("echo", 5),
("foxtrot", 6),
("golf", 7),
("hotel", 8),
("india", 9),
("juliet", 10),
("kilo", 11),
("lima", 12),
("mike", 13),
("november", 14),
("oscar", 15),
("papa", 16),
("quebec", 17),
("romeo", 18),
("sierra", 19),
("tango", 20),
("uniform", 21),
("victor", 22),
("whiskey", 23),
("xray", 24),
("yankee", 25),
("zulu", 26),
]
let expected =
orderedPairs
.sorted { $0.0 < $1.0 }
.map { "\($0.0): \($0.1)" }
.joined(separator: "\n")

let count = orderedPairs.count
for iteration in 0 ..< 30 {
var dictionary: [String: Int] = [:]

// Rotate the pairs to test different permutations
let offset = iteration % count
let rotatedPairs = orderedPairs[offset...] + orderedPairs[..<offset]
for (key, value) in rotatedPairs {
dictionary[key] = value
}

let result = String(data: try encoder.encode(dictionary), encoding: .utf8)!
#expect(result == expected)
}
}

// MARK: - Nested Objects

@Test func deepNestedObjects() async throws {
Expand Down