You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: getting-started/how-retrieval-works/README.md
+5-3Lines changed: 5 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,11 +5,13 @@ description: >-
5
5
6
6
# How retrieval works
7
7
8
-
This section covers how Filecoin retrieval works, from finding providers to fetching content.
8
+
This section covers the implementation details behind Filecoin retrieval: finding providers, choosing a retrieval path, and fetching CID-addressed content.
9
+
10
+
For a high-level overview of managed and direct retrieval paths, see [Retrieval](../what-is-filecoin/retrieval.md).
9
11
10
12
## Table of contents
11
13
12
-
*[Basic retrieval](basic-retrieval.md)— retrieve data with a client like Lassie
13
-
*[Serving retrievals](serving-retrievals.md)— how retrieval deals work and how the indexer fits in
14
+
*[Basic retrieval](basic-retrieval.md)- fetch CID-addressed data with Lassie and work with CAR output
15
+
*[Serving retrievals](serving-retrievals.md)- understand provider advertisements, IPNI, and storage-provider retrieval endpoints
14
16
15
17
[Was this page helpful?](https://airtable.com/apppq4inOe4gmSSlk/pagoZHC2i1iqgphgl/form?prefill\_Page+URL=https://docs.filecoin.io/getting-started/how-retrieval-works)
Copy file name to clipboardExpand all lines: getting-started/how-retrieval-works/basic-retrieval.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,9 +14,9 @@ Lassie is a simple retrieval client for IPFS and Filecoin. It finds and fetches
14
14
lassie fetch <CID>
15
15
```
16
16
17
-
Lassie also provides an HTTP interface for retrieving IPLD data from IPFS and Filecoin peers. Developers can use this interface directly in their applications to retrieve the data.
17
+
Lassie also provides an HTTP interface for retrieving IPLD data from IPFS and Filecoin peers. Developers can use this interface directly in their applications to retrieve data by CID.
18
18
19
-
Lassie fetches content in content-addressed archive (CAR) form, so in most cases, you will need additional tooling to deal with CAR files. Lassie can also be used as a library to fetch data from Filecoin from within your application. Due to the diversity of data transport protocols in the IPFS ecosystem, Lassie is able to use the Graphsync or Bitswap protocols, depending on how the requested data is available to be fetched.
19
+
Lassie fetches content in content-addressed archive (CAR) form, so in most cases, you will need additional tooling to work with CAR files. Lassie can also be used as a Go library. It retrieves CID-addressed IPLD data over the available protocols advertised for that content, including HTTP, Bitswap, or Graphsync depending on provider support. Use provider or service `/piece` endpoints when you need to retrieve a whole PieceCID.
Copy file name to clipboardExpand all lines: getting-started/how-retrieval-works/serving-retrievals.md
+6-8Lines changed: 6 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,25 +9,25 @@ description: >-
9
9
10
10
### The indexer
11
11
12
-
When a storage deal is originally made, the client can opt to make the data publicly discoverable. If this is the case, the storage provider must publish an advertisement of the storage deal to the Interplanetary Network Indexer (IPNI). IPNI maps a CID to a storage provider (SP). This mapping allows clients to query the IPNI to discover where content is on Filecoin.
12
+
When data should be publicly discoverable, the storage provider publishes an advertisement to the InterPlanetary Network Indexer (IPNI). IPNI maps content identifiers to storage providers and retrieval metadata so clients can discover where content is available.
13
13
14
-
The IPNI also tracks which data transfer protocols you can use to retrieve specific CIDs. Currently, Filecoin SPs have the ability to serve retrievals over Graphsync, Bitswap, and HTTP. This is dependent on the SP setup.
14
+
IPNI can also include the retrieval protocols or endpoints a provider advertises for specific CIDs. Filecoin storage providers may serve retrievals over HTTP, Bitswap, Graphsync, or service-specific endpoints depending on their software and configuration.
15
15
16
16
### Retrieval process
17
17
18
18
If a client wants to retrieve publicly available data from the Filecoin network, then they generally follow this process.
19
19
20
20
#### Query the IPNI
21
21
22
-
Before the client can submit a retrieval deal to a storage provider, they first need to find which providers hold the data. To do this, the client sends a query to the Interplanetary Network Indexer.
22
+
Before the client can retrieve from a storage provider, they first need to find which providers hold the data. To do this, the client sends a query to the InterPlanetary Network Indexer.
23
23
24
24
#### Select a provider
25
25
26
-
Assuming the IPNI returns more than one storage provider, the client can select which provider they’d like to deal with. Here, they will also get additional details (if needed) based on the retrieval protocol they want to retrieve the content over.
26
+
Assuming IPNI returns more than one storage provider, the client can select which provider to retrieve from. Here, they will also get additional details based on the retrieval path they want to use.
27
27
28
28
#### Initiate retrieval
29
29
30
-
The client then attempts to retrieve the data from the SP over Bitswap, Graphsync, or HTTP. Note that currently, clients can only get full-piece retrievals using HTTP.
30
+
The client then retrieves the data from the storage provider over one of the advertised paths. HTTP retrieval is the common path for whole-piece `/piece/{pieceCid}` retrieval. Providers that index and advertise IPFS CIDs can also expose `/ipfs/{cid}` style retrieval.
31
31
32
32
#### Finalize the retrieval
33
33
@@ -41,9 +41,7 @@ Different retrieval workflows use different tools:
41
41
| --- | --- |
42
42
| Serving PDP retrievals as a storage provider | Use [Curio](https://curiostorage.org/) for PDP retrievals. |
43
43
| Serving PoRep retrievals as a storage provider | Use [Boost](https://boost.filecoin.io/) to serve retrievals. Boost supports Graphsync retrievals by default, and storage providers can run [`booster-http`](https://boost.filecoin.io/http-retrieval) for HTTP retrievals when configured. |
44
-
| Retrieving data as a client | Use [Lassie](https://github.com/filecoin-project/lassie) to fetch content from Filecoin and IPFS using the best available retrieval path. |
44
+
| Retrieving data as a client | Use [Lassie](https://github.com/filecoin-project/lassie) to fetch CID-addressed content from Filecoin and IPFS using the best available retrieval path. For whole-piece retrieval, use provider or service `/piece` endpoints. |
45
45
| Retrieving application data with Filecoin Onchain Cloud | See the [Filecoin Onchain Cloud retrieval docs](https://docs.filecoin.cloud/core-concepts/retrieval/) for maintained Synapse SDK retrieval flows. |
46
46
47
-
48
-
49
47
[Was this page helpful?](https://airtable.com/apppq4inOe4gmSSlk/pagoZHC2i1iqgphgl/form?prefill\_Page+URL=https://docs.filecoin.io/getting-started/how-retrieval-works/serving-retrievals)
Retrieval is how users fetch content from Filecoin storage providers, IPFS, and Filecoin-backed storage services.
4
+
---
5
+
6
+
# Retrieval
7
+
8
+
Retrieval means fetching stored data back from Filecoin. The right retrieval path depends on how the data was stored, which identifiers you have, and whether you want a managed service API or direct storage-provider retrieval.
9
+
10
+
Use this page as a starting point. For implementation details, see the [How retrieval works](../how-retrieval-works/README.md) section.
11
+
12
+
## Common retrieval paths
13
+
14
+
### Retrieve from Filecoin Onchain Cloud
15
+
16
+
Use the [Filecoin Onchain Cloud retrieval docs](https://docs.filecoin.cloud/core-concepts/retrieval/) when your data was stored through Filecoin Onchain Cloud or the Synapse SDK. Those docs cover the current FOC retrieval paths, SDK flows, and service-specific options.
17
+
18
+
### Retrieve from Fil One
19
+
20
+
[Fil One](https://docs.fil.one/) is an S3-compatible object storage service backed by Filecoin. If your data is stored in Fil One, retrieve it with the same S3-compatible tools, SDKs, or application paths you use for object storage.
21
+
22
+
### Retrieve directly from Filecoin storage providers
23
+
24
+
Use direct storage-provider retrieval when your data was stored through direct deal making or another provider-specific workflow and you need to fetch it from the providers that hold it.
25
+
26
+
Direct retrieval usually starts with either a PieceCID or an IPFS CID:
27
+
28
+
- For a PieceCID, use Filecoin deal, sector, storage-service, or provider metadata to identify a provider that stores the piece, then retrieve from the provider's HTTP `/piece/{pieceCid}` endpoint when available.
29
+
- For an IPFS CID, use content routing such as the [InterPlanetary Network Indexer](https://cid.contact/) to find providers that advertised the CID. Some providers expose an `/ipfs/{cid}` endpoint for IPFS-style retrieval.
30
+
31
+
Use [Lassie](https://github.com/filecoin-project/lassie) for CID-based retrieval from Filecoin and IPFS. Lassie can discover providers for advertised CID content and fetch it with `lassie fetch <CID>`, or run as an HTTP daemon for `/ipfs/{cid}` requests. For whole-piece retrieval by PieceCID, use a provider or service path that supports `/piece`.
32
+
33
+
## Where to go next
34
+
35
+
-[Filecoin Onchain Cloud retrieval](https://docs.filecoin.cloud/core-concepts/retrieval/) covers retrieval for data stored through Filecoin Onchain Cloud.
36
+
-[Fil One docs](https://docs.fil.one/) cover retrieval through S3-compatible object storage APIs.
37
+
-[Basic retrieval](../how-retrieval-works/basic-retrieval.md) covers fetching CID-addressed data with Lassie.
38
+
-[Serving retrievals](../how-retrieval-works/serving-retrievals.md) explains provider advertisements, IPNI, and retrieval protocols.
39
+
40
+
[Was this page helpful?](https://airtable.com/apppq4inOe4gmSSlk/pagoZHC2i1iqgphgl/form?prefill_Page+URL=https://docs.filecoin.io/getting-started/what-is-filecoin/retrieval)
0 commit comments