Skip to content
This repository was archived by the owner on May 15, 2024. It is now read-only.

Revise semantics of out of band deal data transfer #47

Open
masih opened this issue Aug 9, 2023 · 5 comments
Open

Revise semantics of out of band deal data transfer #47

masih opened this issue Aug 9, 2023 · 5 comments
Labels

Comments

@masih
Copy link
Member

masih commented Aug 9, 2023

Boost "transfer params" is the primitive used to convey information about how a CAR is transferred from ISV to an SP as part of deal execution. This primitive is only permitted for online deals. For offline (also referred to as out of band ) deals no such standardisation exists.

One of the key objectives of Project Motion is to make it straightforward for ISVs to handle deal lifecycle end-to-end. Therefore, lack of standardised interfaces for out of band data transfer poses a challenge on how to encapsulate complexity for the ISVs. Further, out of band deals are the preferred method of deal making/data transfer for the SPs. Because, it gives them full control over their data ingestion flow in order to increase efficiency.

For the purposes of MVP, a documentation-only approach seems to suffice, where some human intervention is needed to then facilitate out of band data transfer. For example:

  • There will be a configurable directory in which CAR files are written, named by their piece CID.
  • It is the responsibility of the ISV to set up transfer of CAR files from that directory for the SPs.
    • e.g. set up nginx static content server with some credentials that are pre-communicated to the SPs.
  • Once the deal appears on chain as published, Motion would delete the file from the directory.
@masih masih added the MVP This is included within the MVP label Aug 9, 2023
@LaurenSpiegel
Copy link
Collaborator

Can we go a step beyond documentation and actually include this setup in the automated deployment for motion?

@angelo-schalley-p
Copy link

I agree with @LaurenSpiegel this should be included.

@TorfinnOlsen TorfinnOlsen added the M2.0 Motion API Storage to Filecoin label Aug 11, 2023
@masih
Copy link
Member Author

masih commented Aug 28, 2023

Changing the label to m3.0 since milestone assignment is inconsistent with the label. Let us know if this is needed for M2.0 @celine3650 .

@masih masih added M3.0 Motion Retrievals MVP and removed M2.0 Motion API Storage to Filecoin labels Aug 28, 2023
@masih masih removed M3.0 Motion Retrievals MVP MVP This is included within the MVP labels Oct 5, 2023
@masih masih removed this from the M3.0 Motion retrievals MVP milestone Oct 5, 2023
@masih
Copy link
Member Author

masih commented Oct 5, 2023

@celine3650 this is unlikely to be delivered for Beta considering the timelines and the scope of work which will impact boost.

Product input is needed to determine proirity.

@masih masih added the P3 label Oct 5, 2023
@rvagg
Copy link
Member

rvagg commented Nov 28, 2023

Being able to configure boost to not trigger a download on receiving transfer params but be able to use it as an informational reference for their own fetch-and-onboard workflow may be a good option here because it provides us, as a client, with the ability to be flexible about the piece URL and not rely on out-of-band communication to set that up (which makes dynamic URLs difficult, e.g. fetch from another SP because I no longer have it locally).

Coupled with a special directory (or some other automatic onboarding once they have the piece) they could build straightforward workflows where they are in control but minimize (and eliminate) human interaction.

They would need to be able to inspect the proposals programatically to get at the transfer params though. Hopefully that's not hard with the GraphQL endpoint.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

5 participants