Commit b0eea3d
Add version byte to ClusterConfig and ReplicationHistory serialization (#1778)
* Add version byte to ClusterConfig serialization with gossip validation
Add ClusterConfigVersion constant (version 1) to ClusterConfig that is
serialized as the first byte of the config payload. On deserialization,
the version is validated and an InvalidDataException is thrown on
mismatch.
All gossip paths now peek the version byte before full deserialization
to reject incompatible payloads early with a warning:
- NetworkClusterGossip (receive path)
- GarnetServerNode.GossipAsync (response path)
- TryMeetAsync caller (meet response path)
- ReplicaFailoverSession (failover gossip response path)
Also fix SerializeSlotMap to use relative stream position instead of
hardcoded position 0 for the segment count header.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add version byte to ReplicationHistory serialization
Add ReplicationHistoryVersion constant (version 1) to ReplicationHistory
that is serialized as the first byte of the payload. On deserialization,
the version is validated and an InvalidDataException is thrown on
mismatch.
RecoverReplicationHistory handles version mismatches gracefully by
catching the exception, logging a warning, and reinitializing fresh
state. This is safe because replication history is re-negotiated on
the next primary/replica sync.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add magic prefix to ClusterConfig and ReplicationHistory serialization
Replace the single leading version byte with a 2-byte magic prefix
followed by the version byte. This eliminates ambiguity with legacy
(pre-version) payloads:
- ClusterConfig uses 'GC' (0x47 0x43) magic. As a little-endian
UInt16 this equals 18243, which exceeds the maximum possible legacy
segmentCount (16384), so it can never collide with an old payload.
- ReplicationHistory uses 'GR' (0x47 0x52) magic, distinguishable
from the legacy format which starts with a 7-bit encoded string
length.
TryPeekVersion now validates both magic bytes before extracting the
version, ensuring legacy payloads are reliably rejected rather than
silently misinterpreted.
Add ClusterConfigLegacyPayloadRejectedTest to verify old-format
payloads are properly rejected by both TryPeekVersion and
FromByteArray.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Dispose connection and track failed stats on meet version mismatch
When a MEET response has an incompatible config version, dispose the
newly-created GarnetServerNode to avoid leaking sockets/resources,
and count the meet request as failed in gossip stats.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Broaden recovery exception handling for ReplicationHistory
Catch EndOfStreamException and IOException in addition to
InvalidDataException during replication history recovery. This
handles truncated or corrupted on-disk payloads gracefully by
reinitializing fresh state instead of crashing the server.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove redundant double deserialization in failover and gossip paths
Pass the already-deserialized ClusterConfig object directly to
TryMerge instead of calling FromByteArray a second time. Fixes
both ReplicaFailoverSession and GarnetServerNode.GossipAsync.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Use default UTF-8 encoding for BinaryWriter in serializers
Replace explicit Encoding.ASCII with the default (UTF-8) in
ClusterConfigSerializer and ReplicationHistory BinaryWriters.
The encoding parameter only affects string serialization (ReadString/
Write(string)), not integer types. ASCII silently replaces non-ASCII
characters with '?' causing data corruption. The BinaryReader in
ClusterConfigSerializer was already using the default UTF-8, creating
a writer/reader encoding asymmetry.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Improve magic prefix documentation for backwards compatibility
Expand XML doc comments on ClusterConfigMagic and
ReplicationHistoryMagic to explain:
- The human-readable prefix values ('GC' = Garnet Config,
'GR' = Garnet Replication)
- Why a magic prefix is needed for backwards compatibility
with the legacy pre-versioned format
- How each magic value is guaranteed to never collide with
valid legacy payload headers
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Use ReadOnlySpan<byte> for magic prefix constants
Replace static readonly byte[] with static ReadOnlySpan<byte>
properties backed by UTF-8 string literals ('GC'u8 / 'GR'u8).
This avoids heap allocation — the data is embedded directly in
the assembly as a compile-time constant.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Change ClusterConfig and ReplicationHistory version from byte to int
Update serialization version fields from byte (1 byte) to int (4 bytes)
for both ClusterConfig and ReplicationHistory. This changes the binary
layout to: 2-byte magic word + 4-byte int version.
- ClusterConfigVersion: byte -> int
- ReplicationHistoryVersion: byte -> int
- TryPeekVersion: reads 4 bytes via BitConverter.ToInt32
- FromByteArray: uses ReadInt32() with updated length checks
- Tests: updated to use BitConverter for version corruption/validation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove magic prefix from ClusterConfig and ReplicationHistory serialization
Drop the 2-byte magic prefix ("GC"/"GR") from both formats since legacy
(pre-versioned) format support is not needed. The binary layout is now
simply: version int (4 bytes) + payload. This reduces gossip message
overhead by 2 bytes per exchange.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Change version field from int to byte for ClusterConfig and ReplicationHistory
Revert version type from int (4 bytes) to byte (1 byte) for both
ClusterConfig and ReplicationHistory serialization. The binary layout
is now: version byte (1 byte) + payload, minimizing wire overhead.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 6bc8124 commit b0eea3d
8 files changed
Lines changed: 219 additions & 49 deletions
File tree
- libs/cluster
- Server
- Failover
- Gossip
- Replication
- Session
- test/Garnet.test.cluster
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
43 | 49 | | |
44 | 50 | | |
45 | 51 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | | - | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
11 | 28 | | |
12 | 29 | | |
13 | 30 | | |
14 | 31 | | |
15 | 32 | | |
16 | 33 | | |
17 | | - | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
18 | 38 | | |
19 | 39 | | |
20 | 40 | | |
| |||
60 | 80 | | |
61 | 81 | | |
62 | 82 | | |
| 83 | + | |
63 | 84 | | |
64 | 85 | | |
65 | 86 | | |
| |||
93 | 114 | | |
94 | 115 | | |
95 | 116 | | |
96 | | - | |
| 117 | + | |
97 | 118 | | |
98 | | - | |
| 119 | + | |
99 | 120 | | |
100 | 121 | | |
101 | 122 | | |
| |||
108 | 129 | | |
109 | 130 | | |
110 | 131 | | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
111 | 139 | | |
112 | 140 | | |
113 | 141 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
219 | | - | |
220 | 219 | | |
221 | | - | |
222 | | - | |
223 | | - | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
224 | 225 | | |
225 | | - | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
226 | 235 | | |
227 | 236 | | |
228 | 237 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
189 | 189 | | |
190 | 190 | | |
191 | 191 | | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
192 | 200 | | |
193 | 201 | | |
194 | 202 | | |
195 | 203 | | |
196 | | - | |
| 204 | + | |
197 | 205 | | |
198 | 206 | | |
199 | 207 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
186 | 186 | | |
187 | 187 | | |
188 | 188 | | |
189 | | - | |
190 | | - | |
191 | | - | |
| 189 | + | |
192 | 190 | | |
193 | | - | |
194 | | - | |
195 | | - | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
196 | 207 | | |
197 | | - | |
| 208 | + | |
198 | 209 | | |
199 | | - | |
200 | | - | |
201 | | - | |
202 | | - | |
203 | | - | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
204 | 216 | | |
205 | 217 | | |
206 | 218 | | |
| |||
Lines changed: 25 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
7 | 6 | | |
8 | 7 | | |
9 | 8 | | |
| |||
14 | 13 | | |
15 | 14 | | |
16 | 15 | | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
17 | 22 | | |
18 | 23 | | |
19 | 24 | | |
| |||
43 | 48 | | |
44 | 49 | | |
45 | 50 | | |
46 | | - | |
| 51 | + | |
47 | 52 | | |
| 53 | + | |
48 | 54 | | |
49 | 55 | | |
50 | 56 | | |
| |||
59 | 65 | | |
60 | 66 | | |
61 | 67 | | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
62 | 76 | | |
63 | 77 | | |
64 | 78 | | |
| |||
105 | 119 | | |
106 | 120 | | |
107 | 121 | | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
113 | | - | |
114 | | - | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
115 | 131 | | |
116 | 132 | | |
117 | 133 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
385 | 385 | | |
386 | 386 | | |
387 | 387 | | |
388 | | - | |
389 | | - | |
390 | | - | |
391 | | - | |
| 388 | + | |
| 389 | + | |
392 | 390 | | |
393 | | - | |
394 | | - | |
395 | | - | |
396 | | - | |
397 | | - | |
398 | | - | |
399 | | - | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
400 | 399 | | |
401 | | - | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
402 | 414 | | |
403 | | - | |
404 | | - | |
405 | | - | |
406 | | - | |
| 415 | + | |
| 416 | + | |
407 | 417 | | |
408 | | - | |
409 | | - | |
410 | 418 | | |
411 | 419 | | |
412 | 420 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
160 | 243 | | |
161 | 244 | | |
0 commit comments