Pagination of Porter responses

[derekpierre]

Related to nucypher/nucypher#2703.

If the responses for Alice endpoints /get_ursulas and potentially /revoke can be too large, we can think about paginating the response. See nucypher/nucypher#2666 (comment) for some context.

[cygnusv]

@derekpierre is this done in nucypher/nucypher@f384c7e?

[derekpierre]

@derekpierre is this done in f384c7e?

This issue is still outstanding. The referenced commit is related to pagination of the list of stakers obtained from the smart contract i.e. the back end call. However, this issue is specific to the front end side (endpoint response to the request) i.e. if the user's request asks for something like 500 ursulas, do we return all 500 in a single large response result (as we do now), or do we limit the number that can be returned in a single response and allow for follow-on requests to obtain the remainder of ursulas. One of the concerns here is having to cache sampling results for a period of time as opposed to being totally stateless - see nucypher/nucypher#2666 (comment).

[derekpierre]

To clarify, this didn't seem absolutely necessary for an MVP. Do you feel differently?

[cygnusv]

@derekpierre is this done?

[derekpierre]

No.

This issue is around the question of what if someone wants to sample for a large quantity of nodes - do we return all the information in one response (which is what we do now), or split the response into paginated responses.

For example, if someone called getI_ursulas and wanted 200 ursulas, does the response contain all information for the 200 ursulas, or do we paginate the response in batches of 50. Note Porter would still have to sample for the full 200 nodes , and then save state, to respond to requests for subsequent response pages by the user. The issue here being that Porter has to save some state for subsequent responses to the original request.