[FEATURE] Pluggable shared state storage for Circuit Breaker, Rate Limiter, and Retry Budget Problem

[sector119]

Problem
pyresilience stores all stateful component state in process memory. This works correctly for single-instance deployments, but breaks in any horizontally scaled environment — Kubernetes, Docker Swarm, or multiple workers behind a load balancer.

Concrete failure modes:
Circuit Breaker — each replica maintains its own failure counter. With failure_threshold=5 and 5 replicas, the service absorbs up to 25 failures before any circuit opens. The protection is effectively divided by replica count.

Rate Limiter — with max_calls=100 and 5 replicas, the actual rate sent to the downstream is 500 calls/period. The external API sees no limiting at all.

Retry Budget — the budget exists specifically to prevent retry storms cluster-wide. Per-process budgets multiply by replica count, defeating the entire purpose.

It would be nice to introduce a StateBackend protocol that stateful components accept as an optional dependency. The default (in-memory) behavior is unchanged. Users running multi-node deployments inject a shared backend.

To keep zero-dependency constraint — the Redis backend could live in pyresilience.backends.redis as an optional contrib module, keeping the core package dependency-free.

[AhsanSheraz]

Thanks for the great writeup! You're totally right about the problem. Right now everything that keeps state circuit breaker, rate limiter, retry budget, cache, bulkhead stores it inside a single process. So if you run 5 replicas, you basically get 5 separate copies of that state. Your limits and budgets end up multiplied by the number of replicas, exactly like you said.

And to be clear: there's no way to share this state today. Whatever we build here is brand new, not a small tweak.

Before I lock onto a design, one question for you: what were you picturing for the shared backend? A simple key/value style thing (get / set / increment / expire), or something a bit smarter per pattern? That answer changes a lot of the tradeoffs below.

Here's how I'm thinking about it right now:

Option 1: One shared StateBackend (your idea)

One simple interface that every component talks to, with a Redis version living in an optional package so the core stays dependency-free. This is the cleanest and most flexible.

The one catch: things like rate limiting need the "read → update → save" step to happen as a single atomic action. Over a plain key/value interface that's easy to get slightly wrong, and then it quietly lets a few extra calls through. Doable, just needs care.

Option 2: A tailored backend per pattern

Instead of one generic interface, give each pattern its own small interface and let the Redis version use Redis's own atomic tricks (Lua scripts) built for that exact job. It's more code, but each pattern ends up correct and fast. You'd pass it in like RateLimiterConfig(store=RedisRateLimitStore(...)), and if you pass nothing you get today's in-memory behavior. This is the one I lean toward for correctness.

Option 3: Light-touch syncing (no per-call network hit)

Instead of hitting Redis on every single call, replicas just share their counters with each other every so often. No hard dependency, no slowdown on the hot path. The downside is the numbers are "close enough" rather than exact fine for circuit breakers, weaker for strict rate limits. Cheapest to ship and honest about what it can and can't do.

Two things any version must keep

1. Default stays in-memory and zero-dependency. Redis is always optional, never required.
2. We need a clear rule for when Redis is down should an open circuit breaker still block traffic (safe) or let it through (keeps you running)? That needs to be an explicit setting, not an accident.

Whatever we pick, I'd start small: get it working for the three patterns that need it most rate limiter, retry budget, and circuit breaker and leave the others for later.

---

Would you like to contribute this? If yes, the easiest first step is a quick sketch of the interface for just the rate limiter (it's the simplest one to reason about). Once we're happy with that shape, the rest follows the same pattern. Happy to work through the tricky bits with you.