You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
C++ static initializers that allocate memory aren't really handled by the Redis loadable module machinery. The problem is that the static initializers are executed BEFORE the call to the ...Module_OnLoad which provides the address of ...Module_Allocate. Nor does the Valkey core export it's own internal malloc function. Thus during the execution of the static initializers some mechanism must be provided to perform memory allocations.
The current codebase has a fairly complicated mechanism that uses a private version of malloc (private-malloc) and maintains a separate hash set to track allocations done using the private-malloc so that subsequent free operations can be directed to the correct allocator. This incurs additional overhead for every memory free operation as the tracking machinery is globally locked to perform a hash table lookup and occasionally removal.
Two alternative solutions to this overhead have been suggested:
Have the Valkey core export zmalloc, zfree, etc. so that the dlopen machinery can link those symbols, allowing the static initializers to use those functions directly.
Another solution that doesn't require core changes is to use the linker's wrap facility to swap the compiler provided _init function for a our own no-op function. Now dlopen will invoke our wrapped, empty, initialization function. Later when the core invokes ...Module_OnLoad the code can invoke the renamed _init after the ...Module_Alloc function pointer is known.
The text was updated successfully, but these errors were encountered:
C++ static initializers that allocate memory aren't really handled by the Redis loadable module machinery. The problem is that the static initializers are executed BEFORE the call to the
...Module_OnLoad
which provides the address of...Module_Allocate
. Nor does the Valkey core export it's own internalmalloc
function. Thus during the execution of the static initializers some mechanism must be provided to perform memory allocations.The current codebase has a fairly complicated mechanism that uses a private version of malloc (private-malloc) and maintains a separate hash set to track allocations done using the private-malloc so that subsequent free operations can be directed to the correct allocator. This incurs additional overhead for every memory free operation as the tracking machinery is globally locked to perform a hash table lookup and occasionally removal.
Two alternative solutions to this overhead have been suggested:
Have the Valkey core export
zmalloc
,zfree
, etc. so that thedlopen
machinery can link those symbols, allowing the static initializers to use those functions directly.Another solution that doesn't require core changes is to use the linker's
wrap
facility to swap the compiler provided_init
function for a our own no-op function. Nowdlopen
will invoke our wrapped, empty, initialization function. Later when the core invokes...Module_OnLoad
the code can invoke the renamed_init
after the...Module_Alloc
function pointer is known.The text was updated successfully, but these errors were encountered: