Are pre-mmaped arenas only used when allocating with a heap?

[ericr-eh]

This might be kind of hard to phrase but here goes..

I'm in a situation (on Linux) where I have to pre-map a HUGE amount of memory per NUMA node. I want that memory to then be used (for now) by malloc and free and I do NOT want mimalloc to attempt to mmap more memory later.

I want these arenas to be significantly bigger than 16GB but that doesn't seem to be an option for now.

To accomplish this, I am:

mi_option_set(mi_option_disallow_os_alloc, 1);

....

mmap(...); // no more than 16GB

            if (!mi_manage_os_memory_ex(
                base,
                size,
                true, /* is_committed */
                true, /* is_large */
                true, /* is_zero (mmap zeroes memory) */
                i, /* numa node */
                false /* exclusive */,
                &pool_node_info[i].arena_ids[j])) { // NOTE: I am NOT using these arena_id's for now
                goto fail;
            }

I am later printing the address of the first and last arena per node. The output is:

node 0 virt 0x26c54e600000
node 0 virt 0x26c579c00000
node 1 virt 0x6b88cb600000
node 1 virt 0x6b88f6c00000

But later when trying to do something with the results of posix_memalign, I get the value 0x7fe616141000, which is clearly outside of the above mmap'ed range.

This leads to two questions:

1. Am I misunderstanding the mi_option_disallow_os_alloc option? If the arenas are full, shouldn't I get back a NULL?
2. Why am I not getting something from my mapped arenas? I know I'm not out of memory yet; this is happening early on startup.

Or to put this another way, what should I be doing in order to ensure that mmap is ONLY called at startup and if the arenas fill up I only get NULL back? I thought I had a good handle on the API functions I need but now I'm not so sure.

Thanks!

(EDIT: I'm using version 3.2.7)

[daanx]

ah, that is embarrassing! Your understanding is right -- at the moment asking mimalloc to manage more than a max size arena (16GiB) will ignore the rest of the space. It should give a warning though -- did you run with MIMALLOC_VERBOSE=1 ?
I even have a "todo" comment in the arena allocation section; I will fix this by allocating enough arena's for each memory space. The "disallow_os_alloc" is to disallow direct OS allocation which is used for example for very large objects > 1GiB (mi_option_arena_max_obj_size). mimalloc will still reserve arena's when needed which is why you see the pointers outside the original arenas.
Set mi_option_disallow_arena_alloc to 1 to stop that as well -- now only the initial arenas are available.

to be continued -- I might get to fixing this soon.

[ericr-eh]

Thanks for responding so quickly!

In my case mi_manage_os_memory_ex was failing because mi_manage_os_memory_ex2 returned false at:

  const size_t slice_count = _mi_align_down(size / MI_ARENA_SLICE_SIZE, MI_BCHUNK_BITS);
  if (slice_count > MI_BITMAP_MAX_BIT_COUNT) {  // 16 GiB for now
    // todo: allow larger areas (either by splitting it up in arena's or having larger arena's)
    _mi_warning_message("cannot use OS memory since it is too large (size %zu MiB, maximum is %zu MiB)", size/MI_MiB, mi_size_of_slices(MI_BITMAP_MAX_BIT_COUNT)/MI_MiB);
    return false;
  }

It doesn't look like there's a way to override that size limit. But version 2 doesn't seem to have this limitation (??)

I might try using V2 and create one huge arena per node and one heap per thread. This is a little annoying but not that big of a deal for my use case.

I probably should have mentioned; I was allocating 64MB when it allocated from outside the arena. Nowhere near 1GB and I'm quite certain I haven't filled my arenas by the time I got to this.

I'm a little confused by the documentation for mi_option_disallow_arena_alloc:

1 = do not use arena's for allocation (except if using specific arena id's)

That seems to imply that using malloc or free will never use an arena if it's set to 1? Meaning, malloc and free won't work at all? Or will only work via new mmap?

Some background might be in order. I need an allocator that can do the following:

• I need to be able to mmap ALL of my memory on startup. I do not want the allocator to ever call mmap again. If all arenas are full, any allocation functions should return NULL
• Ideally, I'd like to mmap a single MASSIVE arena per node. Significantly more than 16GB
• I want malloc to work on the current node, which appears to be the existing behavior
• I also need to be able to allocate on specific nodes. This currently appears to be possible only via heaps
• I'd like to have only one heap per node (or one per thread). That way I don't have to try to figure out which heap to use on any given allocation (round robin? try until one fails?)

With the right set of options, can mimalloc do all of that? I'm pretty sure it can but not 100% certain.

Thanks again!

[will-extrahop]

Colleague of @ericr-eh here. 🙂 Just want to clarify a couple things below. Sorry for the random questions. We've been pretty excited about mimalloc but we're still getting familiar with it, and I must admit we've been confused about v2 versus v3 at times, since v3 isn't the main/default branch yet. The v1/v2/v3 annotations that I think were recently added to the docs at https://microsoft.github.io/mimalloc certainly help. 🙂

Ideally we don't want to explicitly use arenas or heaps at all. As @ericr-eh said, we just want to call mi_manage_os_memory() or perhaps mi_manage_os_memory_ex() with our pre-mmaped ranges (one per node) very early during process startup, and then call malloc et al. and have them allocate from the pre-mmaped range for the current node (and if that one is OOM, either fail or fall back to another node). But we also sometimes need to allocate from an explicitly specified node (meaning our pre-mmaped range for that node). That was brought up in #116, but currently it still requires explicitly using arenas and heaps. Would it make sense to create a new issue for that (maybe with a different title)?

As for mi_option_disallow_arena_alloc, we're happy to have very large allocations fail if that's the only effect, but like @ericr-eh said, the doc implied to us that it also broke allocations that don't explicitly pass an arena. Or would they only break if we also passed true for the exclusive arg of mi_manage_os_memory_ex()? We think we want to pass false which I think also negates the PS in this comment? (edit: This comment suggests that it does negate that PS.)

We're also a little unclear if we need to set the default heap (v2) or theap (v3) on every thread as mentioned in #730 to achieve our goals. (We probably need to experiment more though.)

[daanx]

Hi, the latest dev3 has a fix that now allows large OS allocations to be managed by mimalloc; it will use multiple arena's to cover the area so it is still limited to about 2 TiB total (but if you need more you can either increase the MI_ARENA_SLICE_SHIFT to 17 (2x more), or increase MI_MAX_ARENAS). Let me know how it goes.
Also, you can call mi_debug_show_arenas() to see a nice view of all the arenas in use. The large areas should show up as an arena, followed by some sub-arena's in 16 GiB parts.

Moreover, I was indeed wrong about the mi_option_disallow_arena_alloc -- I also added a commit that stops on-demand allocation of arena's if mi_option_disallow_os_alloc is enabled -- so just enabling that should cause malloc to return NULL when there is no more space in any arena.

For now, the only way to allocate specifically to a particular NUMA node is by using explicit heaps that are tied to a specific arena (mi_heap_new_in_arena). Maybe that is ok, but perhaps we should add some API entrypoints for basic allocation to a particular NUMA node.

[will-extrahop]

Wow, that's awesome!! 🙂 2 TiB is enough for now. We should be able to try it shortly!

I see that mi_manage_os_memory_ex() will return the new "parent" arena ID. So when we pass that ID to mi_heap_new_in_arena(), will that heap allocate from all the underlying sub-arenas transparently? That would be nearly ideal since we could just create one (v3) heap per node.

but perhaps we should add some API entrypoints for basic allocation to a particular NUMA node.

With these latest changes, would that provide any functional advantage for our use case compared to creating one heap per node and allocating from those when we need a specific node (and calling plain malloc() et al. when we want mimalloc to pick the node), or would it just be a little more ergonomic?

[ericr-eh]

Ignore my previous comment; I misread something important.

[ericr-eh]

I've had some time to play around with this new dev3 branch. It's definitely getting closer to working for me, thanks @daanx !

Here's what I'm doing now:

• One mmap per node, and pass it to mi_manage_os_memory_ex
• Create one heap per node with mi_heap_new_in_arena
• Call mi_heap_set_numa_affinity with that heap and node id (I think this will force malloc & friends to use that heap when called from a given node; is that correct?)
• Do allocation for the current node with malloc/posix_memalign/etc
• Allocate for a specific node with mi_heap_malloc/mi_heap_malloc_aligned
• Call mi_debug_show_arenas from my registered error handler.

And I'm hitting that error handler during an allocation due to OOM. However the output shows that only the parent arena for the node is full. Everything else is completely empty. @will-extrahop the answer to your question above is currently "NO".

Is there something else I need to do to force mimalloc to use the heap's arena's children? Did I miss some setting or API?

One other thing..

In the output I'm also seeing a 1GB arena 0 associated with numa node -1:
arena 0 at 0x7F6E6FA90000: 16384 slices (1024 MiB), subproc: 0x5576EB3EBB80, numa: -1

What exactly is that? Is it vital or is there some way to avoid it? I assume this happens some time during the constructor but can't tell where. And most importantly: will it be used? If not it's not a huge deal.

Thanks!