This repository was archived by the owner on May 27, 2025. It is now read-only.
fix: gracefully handle redis counters via heartbeat#9
Merged
Conversation
b357a37 to
f237cb2
Compare
ae8a38b to
9b788c9
Compare
9b788c9 to
915de1a
Compare
cody-wang-cb
reviewed
May 5, 2025
cody-wang-cb
left a comment
There was a problem hiding this comment.
Some comments, but this approach makes sense to me overall
cody-wang-cb
approved these changes
May 5, 2025
df031b4 to
9fbb627
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I believe the issue we were seeing with hitting limits is that the websocket-proxy was getting interrupted for reboots and because the connections weren't gracefully terminated, the counter was not decremented. This handles the connection management via heartbeats and periodic cleanup by existing clients.
It adds functionality to reset Redis counters on boot, preventing stale counters from affecting new instances. The core improvement is a heartbeat system using UUID-based instance IDs with background tasks that periodically update instance status and clean up counters from stale instances. These changes provide more reliable rate limiting in distributed environments, automatically recover from instance crashes without manual intervention, and maintain accurate global and per-IP connection counts across multiple proxy instances.