Skip to content

Feat/40 dump restore - #361

Merged
animir merged 6 commits into
animir:masterfrom
man610:feat/40-dump-restore
May 6, 2026
Merged

Feat/40 dump restore#361
animir merged 6 commits into
animir:masterfrom
man610:feat/40-dump-restore

Conversation

@man610

@man610 man610 commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

PR: Add Persistence Support to RateLimiterMemory
Description
This PR introduces the ability to persist and restore the state of RateLimiterMemory. While RateLimiterMemory is primarily designed for high-performance in-memory limiting, there are many use cases where preserving the rate-limiting state across process restarts or deployments is critical.

This implementation adds two primary methods for state management and optional lifecycle hooks for real-time synchronization.

Key Changes

  1. State Serialization & Restoration
    dumpToString(): Serializes the current _storage and _inMemoryBlockedKeys into a JSON string.
    Includes a version field for future-proofing.
    Captures absolute expiration timestamps for all records.
    restoreFromString(serialized): Reconstructs the internal state from a dump.
    TTL Arithmetic: Automatically calculates remaining TTLs based on the expiresAt timestamps. If a key has already expired during the process downtime, it is ignored.
    Timer Reconstruction: Re-initializes expiration timeouts for all restored keys to maintain memory hygiene.
  2. Lifecycle Hooks
    Added two new optional callbacks to RateLimiterMemory constructor:

onUpsert(key, record): Triggered whenever a key's value or expiration is updated (consume, penalty, reward, block).
onDelete(key): Triggered when a key is manually deleted. These hooks enable "fire-and-forget" synchronization to an external persistent store (like Redis or a database) without blocking the synchronous core logic of the memory limiter.
3. Internal Enhancements
Extended MemoryStorage component with getRawAll() and restore() methods to support batch operations required by the persistence logic.
Updated TypeScript definitions in types.d.ts.
Verification Results
A new test file test/RateLimiterMemoryPersistence.test.js has been added covering:

Full dump and restore cycle between different instances.
TTL accuracy across simulated downtime.
Persistence of blocked keys.
Error handling for invalid/corrupt JSON input.
Version mismatch handling.

Comment thread lib/component/MemoryStorage/MemoryStorage.js Outdated
@man610
man610 force-pushed the feat/40-dump-restore branch from 5dcb932 to 3b9b6e6 Compare April 25, 2026 20:16
@man610
man610 requested a review from animir April 30, 2026 07:01
@animir
animir requested a review from Copilot May 1, 2026 19:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds persistence APIs to RateLimiterMemory so its in-memory state can be serialized and restored after process restarts.

Changes:

  • Added dumpToString() / restoreFromString() to RateLimiterMemory for JSON-based state export/import.
  • Added MemoryStorage._restoreRecord() to recreate records (including expiry timers) during restore.
  • Added a new Mocha test suite covering dump/restore behavior and TTL handling; updated TypeScript typings.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.

File Description
lib/RateLimiterMemory.js Implements dumpToString() and restoreFromString() for persistence.
lib/component/MemoryStorage/MemoryStorage.js Adds _restoreRecord() to restore records and reconstruct expiration timeouts.
test/RateLimiterMemoryPersistence.test.js Adds tests for dump/restore, TTL arithmetic, and error handling.
types.d.ts Updates TS declarations for new RateLimiterMemory methods (and fixes some indentation).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/RateLimiterMemory.js Outdated
Comment thread lib/RateLimiterMemory.js Outdated
Comment thread lib/RateLimiterMemory.js Outdated
Comment thread lib/component/MemoryStorage/MemoryStorage.js Outdated
improvment : remove string dump and restore and define type and accept object for restore & give dump in object and define that type as per consumer view

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/RateLimiterMemory.js
Comment thread lib/RateLimiterMemory.js
Comment thread lib/RateLimiterMemory.js
Comment thread test/RateLimiterMemoryPersistence.test.js
@animir

animir commented May 2, 2026

Copy link
Copy Markdown
Owner

@man610 It looks good, very close.

One question is to how developer knows if data restoration is successful?
Should restore function return number of restored records?

If _restoreRecord function returns some codes like invalid, expired and restored then restore function could count total number of each category and return object.

What do you think?

@man610

man610 commented May 2, 2026

Copy link
Copy Markdown
Contributor Author

@animir your thought is Right but first i was just thinking to give Boolean to user,
But as you suggest we can give that object with those keys but i want to extends this functionality
in restore we will accept one more parameter which is optional & by default false parameter like detail response which will include those Four key with object in which one is count and one is array of that counted key
what's you opinion about this ?

@animir

animir commented May 3, 2026

Copy link
Copy Markdown
Owner

@man610 Good idea. Knowing details about records would be helpful for debugging.

@man610

man610 commented May 3, 2026

Copy link
Copy Markdown
Contributor Author

@animir check i have update as per our conversation

@animir

animir commented May 4, 2026

Copy link
Copy Markdown
Owner

@man610 It looks good. The new version will be released soon. Thank you for your patience.

@man610

man610 commented May 4, 2026

Copy link
Copy Markdown
Contributor Author

@animir Thanks for guide me for my first open source contribution & this amazing opportunity

@animir
animir merged commit f1bc96a into animir:master May 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants