Rate Limiting

[thehabes]

There is no limit on traffic from a single IP. A script that uses the API is allowed to run indefinitely. Endpoints used in quick succession or batches are not limited.

[cubap]

We should consider what a reasonable limit is. We've written page loaders that will call for Manifest+Page+Lines with ~100 calls in a few seconds. If a server is trying to build a project, it might 'legitimately' call 1000 times. Also consider that a partner application server may call for several purposes from the same machine.

Perhaps we need to find a distant lid for serious abuse.

• Reads are lightest and least hazardous
• Since/History are heavy calls that should be limited
• Create/Update/Patch cause writes and new data that is more likely to runaway the cost ($ and time)

Proposal:

per	R	S/H	C/U/P
minute	5000	20	1000
hour	100000	200	30000
day	1000000	1000	500000

Questions:

• Should we consider updates and creates differently? The cost is the same, but new data is probably more likely to be an abusive API or something that can be throttled.
• Is it worth the tracking overhead to have long-duration restrictions or is it better to just keep shot-term ones and log violations to find patterns?
• Is there a case for keeping an allow-list for ourselves that can bypass limits or does that reinforce bad practices?