Limit Shortcut API requests to respect rate limits #56
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This library provides a leaky bucket algorithm for rate limiting.
semperos
force-pushed
the
daniel/sc-264542/throttle-entity-creation-rate
branch
2 times, most recently
from
2d98ee6
to
cb583ad
Compare
The pivotal_import.py script is able to print the Shortcut label app_url for the user during the normal course of the import. As of this commit, we do not keep ready access to that app_url either in a JSON or CSV file that we write to disk, so it's not readily available to provide to the user during deletion. So this commit includes a simple print statement that prints one line per entity deleted, conveying both the type (story/epic) and the ID of the Shortcut entity being deleted. If a deletion request to the Shortcut API fails, this commit also prints that error message to the console.
Shortcut's REST API has an advertised rate limit of 200 requests per minute. If 200 or more requests are made within one minute, API clients should expect to receive responses with HTTP status code 429 until enough time has past for the total number of requests made by their API token over the course of the last minute to have dropped below 200. This commit introduces rate limiting on the client (this importer) side, such that users of this pivotal-import tool should never be throttled by Shortcut's API. Since the importer is single-threaded and the rate limiting is handled by an algorithm operating entirely within the Python process' memory, the script should never pause for more than the max_delay set and should never throw an exception related to the rate limit delay being exceeded. It has been configured to throw an exception in case these assumptions do not hold, so that errors are not passed by silently. The rate has been set to 195 requests per minute to keep us comfortably below the rate limit threshold (which is 199 inclusive) without wasting too much possible capacity. The max_delay has been set to 70 seconds to ensure (1) in a case where the max number of requests are made within 1 second, we have a full minute to regain capacity on the Shortcut API side without being throttled by the 200 reqs per minute limit, while also (2) adding a buffer of 10 seconds (1 min == 60 seconds, 60 + 10 == 70 seconds) to account for possible mismatched clock times between the client running this script and the Shortcut API server accepting requests. Co-authored-by: Toby Crawley <[email protected]>
If a user of this importer configures it with a token that is simultaneously making requests to Shortcut from a different script/automation/integration, then that significantly increases the likelihood that they could be throttled by the Shortcut API. This script does not currently provide explicit handling for HTTP 429 requests, and it would be difficult for users to reason about why they were being throttled, so I've added this to the documentation at Toby's suggestion. Co-authored-by: Toby Crawley <[email protected]>
semperos
force-pushed
the
daniel/sc-264542/throttle-entity-creation-rate
branch
from
cb583ad
to
d0cc2d4
Compare
laurenarnett
approved these changes
Apr 10, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
From the primary commit message:
Shortcut's REST API has an advertised rate limit of 200 requests per
minute. If 200 or more requests are made within one minute, API
clients should expect to receive responses with HTTP status code 429
until enough time has past for the total number of requests made by
their API token over the course of the last minute to have dropped
below 200.
This commit introduces rate limiting on the client (this importer)
side, such that users of this pivotal-import tool should never be
throttled by Shortcut's API.
Since the importer is single-threaded and the rate limiting is handled
by an algorithm operating entirely within the Python process' memory,
the script should never pause for more than the max_delay set and
should never throw an exception related to the rate limit delay being
exceeded. It has been configured to throw an exception in case these
assumptions do not hold, so that errors are not passed by silently.
The rate has been set to 195 requests per minute to keep us
comfortably below the rate limit threshold (which is 199 inclusive)
without wasting too much possible capacity.
The max_delay has been set to 70 seconds to ensure (1) in a case where
the max number of requests are made within 1 second, we have a full
minute to regain capacity on the Shortcut API side without being
throttled by the 200 reqs per minute limit, while also (2) adding a
buffer of 10 seconds (1 min == 60 seconds, 60 + 10 == 70 seconds) to
account for possible mismatched clock times between the client running
this script and the Shortcut API server accepting requests.