Handle GitHub API exceptions in feature-links task - #6673
Open
love12yadav wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the feature-links cron/task pipeline so that a single failed GitHub API call (most commonly caused by rate limiting) no longer interrupts an entire batch of link updates.
Addresses #6632
Problem
internals/link_helpers.py's Link.parse() only handles a fixed set of exception types (requests.RequestException, HTTPError, ValueError, APIError).
When GitHub rate-limits requests (403/429), the underlying client may raise an exception outside this handled set. The exception then propagates through _index_feature_links_by_ids() in internals/feature_links.py, which previously had no exception handling.
This caused the /tasks/update-feature-links handler to return a 500 response, triggering Cloud Tasks retries on the same batch of links. The task would repeatedly fail instead of degrading gracefully.
Changes
internals/feature_links.py
Wrapped the per-link Link(...).parse() call inside _index_feature_links_by_ids() with try/except Exception so unexpected failures no longer terminate the entire batch.
Added _http_status_code_from_exception() and _normalise_http_status_code() helpers to extract HTTP status codes from common exception formats (.status_code, .status, .code, and nested .response objects).
Detect 403/429 rate-limit responses from:
raised exceptions, and
Link.parse() results (is_error / http_error_code).
Stop processing gracefully when rate limiting is detected, logging a warning with the number of remaining unprocessed links. Remaining links will be retried during the next scheduled run.
Handle non-rate-limit failures on individual links by recording the error on the corresponding FeatureLinks entity (is_error / http_error_code) and continuing processing of remaining links.
Fixes #6632