Jemalloc quickly OOMs with custom chunk_hooks that opt out of purge

[Benjamin Barg]

Hi all,

I'm writing a set of chunk_hooks that satisfy chunk_alloc requests with a
bump pointer and opt out of dalloc, decommit, and purge. The goal is to
back an arena with 2MB and 1GB huge pages on demand and force jemalloc to
reuse the memory in that space.

I wrote a simple program to test page reuse. As pseudocode:

   1. malloc 1GB of memory using repeated small allocations
   2. free that same 1GB
   3. malloc a small amount of memory again
   4. check whether jemalloc call chunk_alloc again in (3)

I've noticed that when dirty page purging is disabled by setting
MALLOC_CONF="lg_dirty_mult:-1", jemalloc willingly reuses all of the memory
provided by my chunk hooks, but when the option is off, it only reuses
about one chunk's worth (~4Mb).

I would consider this intended behavior (obviously my test program goes
below the default 8:1 minimum active to free page ratio), except that the
man page entry for chunk_purge_t reads

> A chunk purge function conforms to the chunk_purge_t type and *optionally*
discards physical pages

I'm further confused because it seems that API for chunk_hooks_t offers no
mechanism for signaling to jemalloc that the pages were not released.

Basically, I'm asking whether it's intended that opt.lg_dirty_mult must be
set to -1 in order for jemalloc to reuse an arbitrary amount of
chunk_hook-allocated memory.

Very best,
Benjamin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20160615/f3a47594/attachment.html>