Skip to content

Stop relying on filecoinpin.contact for indexing verification, use Curio piece-status endpoint instead #665

Description

@BigLep

Depends on: #664, curio#1405

Summary

filecoin-pin currently polls filecoinpin.contact directly (waitForIpniProviderResults in src/core/utils, exported via filecoin-pin/core/utils) to verify that uploaded content has been indexed and is discoverable. dealbot also consumes this same function today. We're migrating off filecoinpin.contact entirely (see #664), and the replacement logic belongs in filecoin-pin's library so both the CLI and dealbot get it from one place rather than each reimplementing it.

The new polling function does two things, in order:

  1. Poll Curio, not an indexer. Poll Curio's GET /pdp/piece/{pieceCid}/status until it reports synced: true (curio#1405). This replaces today's direct polling of filecoinpin.contact's IPNI-compatible API.
  2. Confirm against cid.contact. Once Curio reports synced, do one confirming query against cid.contact for the root CID (GET https://cid.contact/cid/{rootCid}). By this point it should already be indexed, so this shouldn't trigger cid.contact's negative cache — see [Tracking] Migrate FOC off filecoinpin.contact, use cid.contact only #664's "Intended solution" section for why this ordering matters.

We don't expect step 2 to ever fail once step 1 says synced, but if it does — Curio thinks cid.contact has indexed the content and cid.contact has no record — that's a real anomaly, not routine "still indexing" noise. The library needs to expose that specific case (Curio/cid.contact disagreement) as a distinct, identifiable error or event, separate from "not yet indexed" or "timed out," so callers can react to it deliberately rather than it getting lost inside a generic verification failure.

Steps

  • Identify where filecoin-pin currently polls/queries filecoinpin.contact for indexing status (waitForIpniProviderResults and callers in src/core/).
  • Replace that polling with calls to Curio's GET /pdp/piece/{pieceCid}/status endpoint, waiting for synced/syncedAt, once available (curio#1405).
  • After synced, add the confirming cid.contact query for the root CID.
  • Expose a distinct error/event when Curio reports synced but the cid.contact confirmation query doesn't find the CID (e.g. a dedicated error type or an emitted event distinguishable from "not yet indexed" / "timeout"), so callers can handle it explicitly.
    • Naming/shape (error type vs. event, exact name, fields) is still TBD — needs a decision as part of implementing this issue. @BigLep is happy to discuss/review.
  • filecoin-pin CLI: on that specific mismatch, log an error; consider also collecting a metric/counter for it if the CLI has a metrics path.
  • Update documentation/content-routing-faq.md (and any other docs referencing filecoinpin.contact or its negative-cache behavior) to reflect the new flow through Curio + cid.contact.
  • Remove any remaining filecoinpin.contact references from code, config, and docs.

Notes

Metadata

Metadata

Assignees

Labels

team/filecoin-pin"Filecoin Pin" project is a stakeholder for this work.team/fs-wgFOC working group is a stakeholder for this work, and thus wants to track it on their project board.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions