Crash on startup with security appliance injecting another DLL

[gmnl]

I have the issue, that our windows application is crashing on startup if a 3rd party security appliance (Fast Track Admin by Request) is installed at the computer. The DLL ShellHelper64.dll of the the security appliance is injected on application startup before loading the mimalloc.dll. It seems, that this DLL is doing allocations before the heap manager is redirected to mimalloc. Free the memory after redirection, mimalloc is crashing with an access violation, because it is not able to handle the address of the original heap manager.
I already checked correct position of mimalloc.dll with minject.exe:

minject --list MyApplication.exe
original imported modules (42):
   0: mimalloc.dll
   1: MyDll1.dll
   2: MyDll2.dll
   3: fmt.dll
   4: opencv_core490.dll
   5: opencv_imgcodecs490.dll
   6: opencv_imgproc490.dll
...

Load order as it is shown in the crash dump:

MyApplication.exe
ntdll.dll
kernel32.dll
KERNELBASE.dll
ShellHelper64.dll
rpcrt4.dll
shlwapi.dll
ucrtbase.dll
version.dll
msvcrt.dll
user32.dll
win32u.dll
gdi32.dll
gdi32full.dll
msvcp_win.dll
advapi32.dll
sechost.dll
shell32.dll
WinTypes.dll
combase.dll
ole32.dll
ws2_32.dll
fmt.dll
opencv_core490.dll
MyDlll1.dll
mimalloc.dll
<30 other DLLs>
mimalloc-redirect.dll

When I patch the application with minject.exe, mimalloc_redirect.dll is added before mimalloc.dll and the number of imported modules is increased by 1. The load order changes slightly, but the crash remains.

Is there a way to redirect the heap manager for the whole program before another external DLL is injected? Other ideas how to fix the issue?

SystemInfos:

• MiMalloc 3.0.3
• VS2022 17.6.5
• Win11 25H2

[daanx]

Ah tricky. Try to use the latest v3.2.8 and compile it with MIMALLOC_PAGEMAP_COMMIT=1 ?
(or use in Cmake -DMI_EXTRA_CPPDEFS="MI_DEFAULT_PAGEMAP_COMMIT=1" ).

If that still doesn't work, maybe you can try to edit include/mimalloc/internal.h line 660:

static inline mi_page_t* _mi_ptr_page(const void* p) {
  mi_assert_internal(p==NULL || mi_is_in_heap_region(p));
  #if MI_DEBUG || MI_SECURE || defined(__APPLE__)
  return _mi_checked_ptr_page(p);
  #else
  return _mi_unchecked_ptr_page(p);
  #endif
}

to

static inline mi_page_t* _mi_ptr_page(const void* p) {
  mi_assert_internal(p==NULL || mi_is_in_heap_region(p));
  #if 1 || MI_DEBUG || MI_SECURE || defined(__APPLE__)        // always use check
  return _mi_checked_ptr_page(p);
  #else
  return _mi_unchecked_ptr_page(p);
  #endif
}

this has a minor performance hit but prevents OOM's in the pagemap. I am considering always using checked free mi_cfree in mimalloc-redirect to prevent this by default for any overrides to handle cases like you encounter. TBC. Let me know how it goes.

[gmnl]

I tried to simplify the problem: A simple console application with a simple DLL. The DLL exports a method which returns a std::string. The application calls this method and on destruction of the std::string, the application crashes if FastTrack Admin by Request DLL is injected on startup.

Details:
It seems, that _mi_redirect_entry is not called in the error case. A query to mi_is_redirected() returns false, if starting outside debugger of visual studio (in the case of the crash). The crash happens, if a object is created in another DLL and released in the application. In the DLL, the operator override for new and delete is not active and malloc / free are using their default implementation. In the application, new and delete are redirected to mi_malloc/ mi_free. Then the heaps are mixed as soon as an object is released in the application with delete (-> mi_free) which has been created in the DLL with new (-> malloc default implementation)

So the question is: how is mimalloc_redirect.dll implemented. The source code is still not published. see Issue How to build mimalloc-redirect.dll?
Is there a way to figure out where the hooking of the ucrtbase.dll goes wrong?