Feat/40 dump restore - #361
Conversation
…flexible into feat/40-dump-restore
…flexible into feat/40-dump-restore
5dcb932 to
3b9b6e6
Compare
There was a problem hiding this comment.
Pull request overview
Adds persistence APIs to RateLimiterMemory so its in-memory state can be serialized and restored after process restarts.
Changes:
- Added
dumpToString()/restoreFromString()toRateLimiterMemoryfor JSON-based state export/import. - Added
MemoryStorage._restoreRecord()to recreate records (including expiry timers) during restore. - Added a new Mocha test suite covering dump/restore behavior and TTL handling; updated TypeScript typings.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
lib/RateLimiterMemory.js |
Implements dumpToString() and restoreFromString() for persistence. |
lib/component/MemoryStorage/MemoryStorage.js |
Adds _restoreRecord() to restore records and reconstruct expiration timeouts. |
test/RateLimiterMemoryPersistence.test.js |
Adds tests for dump/restore, TTL arithmetic, and error handling. |
types.d.ts |
Updates TS declarations for new RateLimiterMemory methods (and fixes some indentation). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
improvment : remove string dump and restore and define type and accept object for restore & give dump in object and define that type as per consumer view
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@man610 It looks good, very close. One question is to how developer knows if data restoration is successful? If What do you think? |
|
@animir your thought is Right but first i was just thinking to give Boolean to user, |
|
@man610 Good idea. Knowing details about records would be helpful for debugging. |
|
@animir check i have update as per our conversation |
|
@man610 It looks good. The new version will be released soon. Thank you for your patience. |
|
@animir Thanks for guide me for my first open source contribution & this amazing opportunity |
PR: Add Persistence Support to RateLimiterMemory
Description
This PR introduces the ability to persist and restore the state of RateLimiterMemory. While RateLimiterMemory is primarily designed for high-performance in-memory limiting, there are many use cases where preserving the rate-limiting state across process restarts or deployments is critical.
This implementation adds two primary methods for state management and optional lifecycle hooks for real-time synchronization.
Key Changes
dumpToString(): Serializes the current _storage and _inMemoryBlockedKeys into a JSON string.
Includes a version field for future-proofing.
Captures absolute expiration timestamps for all records.
restoreFromString(serialized): Reconstructs the internal state from a dump.
TTL Arithmetic: Automatically calculates remaining TTLs based on the expiresAt timestamps. If a key has already expired during the process downtime, it is ignored.
Timer Reconstruction: Re-initializes expiration timeouts for all restored keys to maintain memory hygiene.
Added two new optional callbacks to RateLimiterMemory constructor:
onUpsert(key, record): Triggered whenever a key's value or expiration is updated (consume, penalty, reward, block).
onDelete(key): Triggered when a key is manually deleted. These hooks enable "fire-and-forget" synchronization to an external persistent store (like Redis or a database) without blocking the synchronous core logic of the memory limiter.
3. Internal Enhancements
Extended MemoryStorage component with getRawAll() and restore() methods to support batch operations required by the persistence logic.
Updated TypeScript definitions in types.d.ts.
Verification Results
A new test file test/RateLimiterMemoryPersistence.test.js has been added covering:
Full dump and restore cycle between different instances.
TTL accuracy across simulated downtime.
Persistence of blocked keys.
Error handling for invalid/corrupt JSON input.
Version mismatch handling.