Commit 357a36e
committed
Fix non-deterministic fingerprint for nested hashes by sorting
Consider this example:
```JSON
{
"@timestamp": "2023-05-23T23:23:23.555Z",
"@Version": "1",
"beat": {
"hostname": "gnu.example.com",
"name": "gnu.example.com",
"version": "5.2.2"
},
"host": "gnu.example.com"
}
```
Using this filter:
```Logstash
fingerprint {
concatenate_all_fields => true
target => "[@metadata][_id]"
method => "SHA512"
key => "XXX"
base64encode => true
}
```
Here, the order of the `.beat` hash is non-deterministic and the plugin
did not do a deep sort as part of the serialization. This resulted in
different fingerprints for the same event because the order of the three
keys (hostname, name, version) changed randomly in the serialization.
This has been fixed by recursively checking for hashes and serializing
them in sorted order.
Note that this changes the serialization format and thus breaks
backwards compatibility. The old format could be emulated in order to
not break backwards compatibility. Backwards compatibility in this case
means to generate the same fingerprint for the same input.
Closes: #391 parent aa7d522 commit 357a36e
1 file changed
Lines changed: 17 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
104 | 104 | | |
105 | 105 | | |
106 | 106 | | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
107 | 122 | | |
108 | 123 | | |
109 | 124 | | |
| |||
120 | 135 | | |
121 | 136 | | |
122 | 137 | | |
123 | | - | |
124 | | - | |
125 | | - | |
| 138 | + | |
126 | 139 | | |
127 | 140 | | |
128 | | - | |
| 141 | + | |
129 | 142 | | |
130 | 143 | | |
131 | 144 | | |
| |||
0 commit comments