[Tracking] Migrate FOC off filecoinpin.contact, use cid.contact only

[BigLep]

This is about migrating FOC as a whole from using filecoinpin.contact (not just filecoin-pin affordances). The tracking issue needs to live somewhere, and this seemed like the best repo to house it.

Intended solution

Details

• Curio SPs advertise only to cid.contact (no longer to filecoinpin.contact).
• Curio SPs confirm indexing themselves: since Curio already knows the advertisement CID it just announced, it can poll cid.contact's /sync/status/ad/{adCid} endpoint directly to learn whether that specific ad has been fully processed. This endpoint doesn't have the negative-caching problem because it's keyed on the ad CID, not the content CID.
• Curio surfaces that confirmation on its own piece-status endpoint (GET /pdp/piece/{pieceCid}/status, per curio#1405: adding synced/syncedAt fields). Clients like filecoin-pin and dealbot poll that endpoint, not cid.contact, to learn indexing status.
  • The key reason clients can't just poll cid.contact directly: cid.contact's ad-status check is keyed on the advertisement CID, and Curio is the only party that knows the advertisement CID for a given upload. An end client like filecoin-pin only knows the IPFS root CID, and has no structural way to derive the ad CID from it. If such a client polled /cid/{rootCid} on cid.contact before the content was actually indexed, it would trigger the negative-cache issue this whole migration is trying to avoid, hence the need to go through Curio's own status endpoint first, and only fall back to cid.contact once Curio has confirmed the content is ready.
• Only once Curio's endpoint reports the piece as synced do clients query cid.contact for the root CID (GET https://cid.contact/cid/{rootCid}). By then it should already be indexed, so the query shouldn't produce a negative-cache miss.
  • This final cid.contact query is a genuine confirmation step, not a formality: we don't expect Curio's synced signal and cid.contact's own record to ever disagree, but if they do (Curio says synced, cid.contact has no record for the root CID), that's a real anomaly worth surfacing loudly rather than silently retrying or swallowing. filecoin-pin's library should expose this as a distinct, identifiable error/event (separate from "not yet indexed"), and callers (the filecoin-pin CLI, dealbot) are each responsible for logging/metricing it. See the filecoin-pin and dealbot issues below for specifics.

Steps

• Confirm we aren't seeing worse performance from cid.contact than we saw from filecoinpin.contact.
  • Historical filecoinpin.contact numbers: dealbot events-and-metrics docs, ipniVerifyMs — currently not showing up in the BetterStack dashboard. Ideally fix that chart so we can compare against Probelab's cid.contact analysis.
• (ECD 2026-08-28) Determine acceptable way to turn off filecoinpin.contact that doesn't derail PDP SPs that still announce advertisements to filecoinpin.contact (e.g., 200 OK response)
  • [Tracking] Migrate FOC off filecoinpin.contact, use cid.contact only #664 (comment)
• (ECD 2026-08-31) Setup filecoinpin.contact's DNS so FOC working group is self-service to make changes (e.g., use CloudFlare Workers).
• Give a way for Curio clients to determine whether a given CID has been indexed by cid.contact without hitting cid.contact's negative caching issue. Tracked in curio#1405, with related latency work in curio#1426.
• (ECD 2026-08-28) Curio release with this change
  • https://github.com/filecoin-project/curio/releases/tag/v1.28.6
  • Same version as part of [Tracking Issue] 202608 FOC Release FilOzone/filecoin-services#597
• (ECD 2026-08-31) Deploy the transition CloudFlare worker before filecoinpin shuts down
  • [Tracking] Migrate FOC off filecoinpin.contact, use cid.contact only #664 (comment)
  • Domain name resolution change (if Shipyard backend shut down on 2026-09-01)
  • 2026-08-31 update: no longer necessary given Shipyard backend will stay running another couple of weeks.
• (ECD 2026-09-02) Curio PDP SPs all update to this v1.28.6 release.
  • Happening as part of [Tracking Issue] 202608 FOC Release FilOzone/filecoin-services#597
• (ECD 2026-08-31) filecoin-pin affordance updates
  • Update filecoin-pin affordances to use this new Curio method rather than polling filecoinpin.contact.
  • Update docs to no longer describe filecoin-pin as relying on filecoinpin.contact.
  • Dedicated filecoin-pin issue: Stop relying on filecoinpin.contact for indexing verification, use Curio piece-status endpoint instead #665 | PR: feat!: confirm indexing via Curio piece-status #689
  • This shouldn't be merged until Curio PDP SPs all update to the release above.
• (ECD 2026-09-03) Do a filecoin-pin release.
• (ECD 2026-09-03) Update filecoin-pin-website to use the newest filecoin-pin release, including updating its docs (e.g. HTTP_REQUESTS_TO_CURIO.md, which currently describes GET https://filecoinpin.contact/cid/{cid}).
  • fix: handle new indexing-confirmation events from filecoin-pin filecoin-pin-website#236
  • The website's src/hooks/use-ipni-check.ts calls filecoin-pin's waitForIpniProviderResults directly with no explicit ipniIndexerUrl override, so it should pick up the new behavior automatically once the dependency is bumped. This step is mainly about the version bump + doc updates.
• (ECD 2026-09-03) Dealbot updates
  • Update dealbot to stop polling filecoinpin.contact and use the new Curio method instead (may come for free from the filecoin-pin update above).
  • Update dealbot docs to remove filecoinpin.contact references.
  • Dedicated dealbot issue: FilOzone/dealbot#684
• (ECD 2026-09-03) Deploy dealbot to staging and production with these changes.

---

ℹ️ It should be safe to "turn off" filecoinpin.contact at this point. We'll follow the plan in #664 (comment)

---

• (ECD 2026-09-04) Update Curio to stop advertising to filecoinpin.contact.
• (ECD 2026-09-??) Do a Curio release and deploy to SPs to stop using filecoinpin.contact.
  • Must happen after the filecoin-pin release and dealbot deployment above, so we're sure no clients still depend on filecoinpin.contact.
• (ECD 2026-09-02)Full search across the FilOzone org for remaining filecoinpin.contact references.
• (ECD 2026-09-08) Have the IPNI team remove the cid.contact fallback to filecoinpin.contact.
• Transfer filecoinpin.contact domain from IP Shipyard to FOC Working Group.

Why

Details

filecoinpin.contact's current hardware is going away. Rather than stand up and operate replacement infrastructure, we're going to simplify and rely on cid.contact going forward: we believe its reliability and performance are good enough for our needs today.

For context, filecoinpin.contact's backend (Pulsar) doesn't resemble the storetheindex/IPNI deployment behind cid.contact. It was a different system built to work around scaling and operational reliability issues seen with IPNI, exposing an IPNI-compatible API as a compatibility layer rather than a long-term architecture. Standing up and operating a replacement of that infrastructure ourselves is not something we're taking on, so we're consolidating on cid.contact instead.

[beck-8]

Confirm we aren't seeing worse performance from cid.contact than we saw from filecoinpin.contact.

I didn't get any intuitive data here to prove that filecoinpin.contact is better, because it doesn't have the AD interface, and the comparison data should control a single variable. But the data that can be provided here is: cid.contact. From my test results, under normal circumstances it is almost the same as the measurement value in probelab(within 2 seconds). The influencing factor is only the interval between curio publishing AD.

The missing data of the main network is solved in infraPR.

The testnet has data,. but there is still some scheduling time here. (No way to compare horizontally)

So based on the above situation, I want to conclude that cid.contact is not bad

[BigLep]

Nice work @beck-8 . I agree we are seeing data that makes cid.contact "good enough" for now.

Thanks as well for getting https://github.com/FilOzone/infra/pull/371 out. I am assuming that will cause us to have historgram data flowing in prod again? (We can talk more in the PR.)

[beck-8]

Confirm we aren't seeing worse performance from cid.contact than we saw from filecoinpin.contact.
update: Answer Yes

I now get more accurate data (although the comparison dimension may be different). This confirms our above conclusion. @BigLep

• Both instances were run on the same machine, so this isolates the indexer-side difference from network path/geography variables.
• Scheduling interval: PublishInterval = 1s (checkSyncStatus runs once per second).
• cid.contact: measured via /sync/status/ad/{adCid} — asks "has this specific advertisement been fully processed by the indexer," tied to an exact ad_cid.
• filecoinpin.contact: measured via a temporary swap to /cid/{contentCid} — asks "is this content multihash discoverable, does the provider list include me." This checks content findability, not advertisement processing status - not the same dimension as the cid.contact numbers, so treat this as a rough signal, not an apples-to-apples comparison.
• Why is there no test /cid of cid.contact? Because it has a negative cache, the query result will be 404

cid.contact (/sync/status/ad, 6 samples)

Time	ad_cid (truncated)	announced_at	confirmed_at	latency (s)	check_took (s)
09:49:06	baguqeerappend...ola	09:49:05.378	09:49:06.316	0.938	0.199
10:04:06	baguqeeraxynut...wa	10:04:05.596	10:04:06.317	0.721	0.200
10:33:57	baguqeeraarrel...aa	10:33:56.355	10:33:57.205	0.850	0.087
10:48:54	baguqeerajntuq...7a	10:48:53.510	10:48:54.204	0.694	0.086
11:03:49	baguqeeraxv26j...ra	11:03:48.420	11:03:49.208	0.787	0.090
11:19:13	baguqeerasrexg...aq	11:19:12.336	11:19:13.319	0.983	0.202

Mean latency ≈ 0.829s (min 0.694 / max 0.983), mean check_took ≈ 0.144s

filecoinpin.contact (/cid, 8 samples)

Time	ad_cid (truncated)	announced_at	confirmed_at	latency (s)	check_took (s)
10:52:11	baguqeeradjfxq...2a	10:52:08.728	10:52:11.630	2.902	0.208
10:54:05	baguqeeramruo7...bq	10:54:02.959	10:54:05.174	2.215	0.278
10:55:37	baguqeeravdtwj...ca	10:55:34.933	10:55:37.143	2.210	0.275
11:07:25	baguqeeraj47cf...aa	11:07:22.176	11:07:25.130	2.954	0.262
11:08:25	baguqeerapafgz...vq	11:08:21.932	11:08:25.117	3.185	0.248
11:22:11	baguqeerar57rd...hq	11:22:09.180	11:22:11.117	1.937	0.249
11:23:43	baguqeerao5mnz...oa	11:23:40.931	11:23:43.116	2.185	0.249
11:25:17	baguqeeracvhnw...ha	11:25:13.932	11:25:17.118	3.186	0.250

Mean latency ≈ 2.597s (min 1.937 / max 3.186), mean check_took ≈ 0.252s

Takeaways

• filecoinpin.contact's mean latency is roughly 3x higher than cid.contact's (2.6s vs 0.83s), measured from the same host at the same time.
• The single query itself (check_took) is also slower for filecoinpin.contact (0.25s vs 0.14s), so part of the gap is slower confirmation and part is the request/response itself taking longer (network/server-side processing on their end - can't cleanly separate the two from this data).
• Since the two sides aren't checking the same thing (ad-processing confirmation vs. content-findability), some of this 3x gap could reflect a genuine difference in how "done" is defined by each check method, not purely indexer speed. Treat this as a rough signal.

---

other

I still can't find out why there is no data on that panel.

I don't know why. I don't want to watch this for the time being. I have no idea. And the data sources of prod and stag are inconsistent.

On mainnet (Infra Prod), bucket_counts contains actual data after merging (7 out of 9 buckets), yet bucket_bounds results in an empty array 100% of the time—this has been verified across two different metrics (pullRequestAcknowledgementLatencyMs and ipniVerifyMs) and reproduced twice.
Using the same query pattern on calibnet (Infra Staging · Prometheus push), however, bucket_bounds correctly returns an array of numerical values.