Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revise Startup memory allocation machinery #30

Open
allenss-amazon opened this issue Feb 2, 2025 · 0 comments
Open

Revise Startup memory allocation machinery #30

allenss-amazon opened this issue Feb 2, 2025 · 0 comments

Comments

@allenss-amazon
Copy link
Member

C++ static initializers that allocate memory aren't really handled by the Redis loadable module machinery. The problem is that the static initializers are executed BEFORE the call to the ...Module_OnLoad which provides the address of ...Module_Allocate. Nor does the Valkey core export it's own internal malloc function. Thus during the execution of the static initializers some mechanism must be provided to perform memory allocations.

The current codebase has a fairly complicated mechanism that uses a private version of malloc (private-malloc) and maintains a separate hash set to track allocations done using the private-malloc so that subsequent free operations can be directed to the correct allocator. This incurs additional overhead for every memory free operation as the tracking machinery is globally locked to perform a hash table lookup and occasionally removal.

Two alternative solutions to this overhead have been suggested:

  1. Have the Valkey core export zmalloc, zfree, etc. so that the dlopen machinery can link those symbols, allowing the static initializers to use those functions directly.

  2. Another solution that doesn't require core changes is to use the linker's wrap facility to swap the compiler provided _init function for a our own no-op function. Now dlopen will invoke our wrapped, empty, initialization function. Later when the core invokes ...Module_OnLoad the code can invoke the renamed _init after the ...Module_Alloc function pointer is known.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant