You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: src/routing/kad-dht.md
+175-60
Original file line number
Diff line number
Diff line change
@@ -6,10 +6,11 @@ description: >
6
6
InterPlanetary File System (IPFS). It extends the libp2p Kademlia DHT
7
7
specification, adapting and adding features to support IPFS-specific
8
8
requirements.
9
-
date: 2022-08-26
9
+
date: FIXME
10
10
maturity: reliable
11
11
editors:
12
12
- name: Guillaume Michel
13
+
url: https://guillaume.michel.id
13
14
github: guillaumemichel
14
15
affiliation:
15
16
name: Shipyard
@@ -28,6 +29,10 @@ requirements.
28
29
29
30
FIXME:
30
31
32
+
Distributed Key-Value Store
33
+
34
+
Goal of DHT is to find the closest peers to some key (in a specific geometry). Once this routing to the closest nodes is possible, nodes can interact with these nodes in various ways, including in asking them to store and serve data.
35
+
31
36
### Relation to [libp2p kad-dht](https://github.com/libp2p/specs/tree/master/kad-dht)
32
37
33
38
The IPFS Kademlia DHT specification is a specialization of the libp2p Kademlia DHT.
@@ -39,112 +44,218 @@ in the [Amino DHT](#relation-to-the-amino-dht). If you're designing a new
39
44
Kademlia-based DHT for use with IPFS, some details in this specification may
40
45
appear overly specific or prescriptive.
41
46
42
-
### Relation to the [Amino DHT](https://blog.ipfs.tech/2023-09-amino-refactoring/#why-amino)
47
+
### Relation to the [Amino DHT](#amino-dht)
43
48
44
-
The Amino DHT is the swarm of peers also referred to as the _Public IPFS DHT_.
45
-
It implements the IPFS Kademlia DHT specification and uses the protocol
46
-
identifier `/ipfs/kad/1.0.0`. The Amino DHT can be joined by using the [Amino
Nodes participating in the [Amino DHT Swarm](#amino-dht) MUST implement the
50
+
IPFS Kademlia DHT specification. The IPFS Kademlia DHT specification MAY be
51
+
used in other DHT swarms as well.
49
52
50
-
The Amino DHT is utilized by multiple IPFS implementations, including
51
-
[`kubo`](https://github.com/ipfs/kubo) and
52
-
[`helia`](https://github.com/ipfs/helia). Multiple DHT swarms can coexist and
53
-
nodes MAY participate in multiple DHT swarms. DHT swarms can be either public
54
-
or private.
53
+
## DHT Swarms
55
54
56
-
Note that there could be multiple distinct DHT swarms using the same protocol
57
-
identifier as long as they don't have any common peers. This practice is
58
-
discouraged as networks will immediately merge if they enter in contact. Each
59
-
DHT swarm SHOULD have a dedicated protocol identifier.
55
+
A DHT swarm is a group of interconnected nodes running the IPFS Kademlia DHT protocol, collectively identified by a unique protocol identifier. IPFS nodes MAY participate in multiple DHT swarms simultaneously. DHT swarms can be either public or private.
60
56
61
-
## Protocol Parameters
57
+
###Protocol Identifier
62
58
63
-
FIXME: move parameters to appropriate sections
59
+
All nodes participating in the same DHT swarm MUST use the same libp2p protocol
60
+
identifier. The libp2p protocol identifier uniquely identifies a DHT swarm. It
61
+
follows the format `/<swarm-prefix>/kad/<version>`, e.g `/ipfs/kad/1.0.0` for
62
+
the Amino DHT protocol version `1.0.0`, or `/ipfs/lan/kad/1.0.0` for a local
63
+
DHT swarm.
64
64
65
-
The IPFS Kademlia DHT defines a number of Client and Server parameters that
66
-
need to be set to ensure the DHT operates correctly as a system.
65
+
Note that there could be multiple distinct DHT swarms using the same libp2p
66
+
protocol identifier as long as they don't have any common peers. This practice
67
+
is discouraged as networks will immediately merge if they enter in contact.
68
+
Each DHT swarm SHOULD have a dedicated protocol identifier.
67
69
68
-
### Protocol Identifier
70
+
### Amino DHT
69
71
70
-
All nodes participating in the same DHT swarm MUST use the same protocol
71
-
identifier. The protocol identifier uniquely identifies a DHT swarm. It follows
72
-
the format `/<swarm-prefix>/kad/<version>`, e.g `/ipfs/kad/1.0.0` for the Amino
73
-
DHT protocol version `1.0.0`, or `/ipfs/lan/kad/1.0.0` for a local DHT swarm.
72
+
The [Amino DHT](https://blog.ipfs.tech/2023-09-amino-refactoring/#why-amino) is
73
+
the swarm of peers also referred to as the _Public IPFS DHT_. It implements the
74
+
IPFS Kademlia DHT specification and uses the protocol identifier
75
+
`/ipfs/kad/1.0.0`. The Amino DHT can be joined by using the [Amino DHT
Nodes MUST NOT be removed from the Routing Table as long as they remain online.
168
+
Therefore, the bucket replacement policy is based on seniority, ensuring that
169
+
the most stable peers are eventually retained in the Routing Table.
107
170
108
-
Implementation specific. Recommendation is `10`
171
+
#### IP Diversity Filter
109
172
110
-
### Resiliency
173
+
SHOULD implement
111
174
112
-
Implementation specific. Recommendation is `3`
175
+
FIXME:
113
176
114
-
### Routing Table Refresh Interval
177
+
### Routing Table Refresh
115
178
116
-
SHOULD `10min`. Only peers that have been seen in the last 10 minutes should remain in the routing table. If peer hasn't been seen recently, try to ping it to see if it's still alive.
179
+
There are several strategies a DHT Server can use to verify that nodes in its
180
+
Routing Table remain reachable. Implementations may choose their own methods,
181
+
provided they avoid serving unresponsive nodes. One recommended strategy is to
182
+
periodically refresh the Routing Table.
117
183
118
-
## DHT Swarm
184
+
DHT Servers SHOULD perform a Routing Table Refresh every `10` minutes. During
185
+
this process, the server sends a ping request to all nodes it hasn’t heard from
186
+
recently (e.g in the last 5 minutes). Any peer that fails to respond MUST be
187
+
removed from the Routing Table.
119
188
120
-
## Routing Table
189
+
After removing unresponsive peers, any buckets that are not full MUST be
190
+
replenished with fresh, online peers. This can be accomplished by either adding
191
+
recently connected peers or by executing a `FIND_NODE` request with a randomly
192
+
generated Peer ID matching the bucket. `FIND_NODE` requests should only be run
193
+
for buckets up to the last non-empty bucket.
121
194
122
-
### Routing Table Refresh
195
+
Finally, the refresh process concludes by executing a `FIND_NODE` request for
196
+
the local node's Peer ID, ensuring the DHT Server maintains up-to-date
197
+
information on its closest peers.
123
198
124
-
### Public addresses
199
+
##Lookup Process
125
200
126
-
### IP Diversity Filter
201
+
Iterative vs Recursive
127
202
128
-
SHOULD implement.
203
+
### Server behavior
129
204
130
-
## Lookup Process
205
+
In public DHT swarms, DHT Servers MUST never respond with private or loopback multiaddresses.
206
+
207
+
Should Server tell Client about Server? And about Client?
208
+
209
+
### Concurrency
210
+
211
+
Implementation specific. Recommendation is `10`
131
212
132
213
### Lookup termination
133
214
134
215
This is hard
135
216
217
+
#### Resiliency
218
+
219
+
Implementation specific. Recommendation is `3`
220
+
136
221
## Peer Routing
137
222
138
223
DHT Clients that want to be routable must make sure they are in the peerstore of the closest DHT servers to their own PeerID.
139
224
140
225
When performing a `FIND_NODE` lookup, the client will converge to the closest nodes in XOR distance to the requested PeerID. These nodes are expected to know the multiaddrs of the target peer. The
141
226
227
+
### Routing to non-DHT Servers
228
+
142
229
### Signed Peer Records
143
230
144
231
## Content Routing
145
232
233
+
### Content Kademlia Identifier
234
+
235
+
sha256
236
+
146
237
### Provider Records
147
238
239
+
#### Provide Validity
240
+
241
+
Provide Validity defines the time-to-live (TTL) of a Provider Record on a DHT
242
+
Server. DHT Servers MUST implement a Provide Validity of `48h`.
243
+
244
+
#### Provider Record Republish Interval
245
+
246
+
Because of the churn in the network, Provider Records need to be republished
247
+
more often than their validity period. DHT Clients SHOULD republish Provider
0 commit comments