Porting jemalloc to shared memory

[Abi Varghese]

Hello Experts,

We have a static allocator based on shared memory in use today (with
predefined memory segment sizes), but there is just too much of internal
fragmentation and our memory requirement in keep on increasing (we have
close to 60-70GB in use today - with practically needing 20-30GB). We tried
several option to live with the current allocator but it is extremely hard
to have a configuration which can work with varying traffic patterns.

To address the problem, I am considering to use a dynamic allocator based
on shared memory. The basic idea is to have the allocator work across
process boundaries and to have the things stored in shared memory to
survive process crashes or restarts.

After analyzing many allocators, I am thinking to port jemalloc to shared
memory. I have done a very high level look into jemalloc with a simple
prototype. In my prototype I am pre-allocating a big shared memory chunk
using a fixed address (which is aligned to chunk size boundary of 4MB).
With this I think jemalloc would be able to manage memory across processes.

I have few queries in general,

1. Currently I am using a fixed address aligned to 4MB boundary. Is there a
way for me to avoid using a fixed address?

2. Is it safe to use a fixed high memory address for my application?

3. Is it possible to make jemalloc work with page alignment for chunks
instead of chunk alignment? If yes, could you suggest any way for this -
without huge rework in jemalloc? My primary idea is to get out of fixed
address (AFAIK, Linux guarantees shared memory segments to be page aligned)

4. I see that tsd cleanup do not work (including arena cleanup) when
process exits. Is there a way to ensure this? Or do I need to introduce
some garbage collection mechanism on top?

Thanks and Regards,
Abi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20160217/62089566/attachment.html>