Transparent Huge Pages
justin.lebar at gmail.com
Mon Feb 20 08:10:32 PST 2012
Hm, upon further consideration...
If you mmap a huge page (say, 1MB), then MADV_DONTNEED a few 4-KB
chunks inside, transparent huge pages should break up the huge page so
it can decommit the parts I asked it to decommit. If it doesn't, that
sounds like a kernel bug to me!
Similarly, if I mmap 1MB, get a huge page, and then touch only a few
bytes in the middle, the kernel shouldn't commit a huge page.
If huge pages is behaving how I expect, I don't see why it would cause
your application to use more memory.
Just to check, you're measuring RSS, not vsize, right?
On Mon, Feb 20, 2012 at 4:59 PM, Justin Lebar <justin.lebar at gmail.com> wrote:
>> jemalloc seems to be prone to transparent huge pages
>> (https://lwn.net/Articles/423584), presumably due to its use of mmap(). In
>> my case (fuse module), the initial memory consumption jumped from ~12M to
>> ~27M. The use of --enable-dss helps a little, bringing the consumption down
>> to ~19M.
>> Did anyone else experienced similar behavior? Is there an easy way of
>> avoiding transparent huge pages for jemalloc'ed memory? The only workaround
>> that comes to my mind is a malloc wrapper that runs madvise(...,
>> MADV_NOHUGEPAGE) on every newly allocated chunk.
> You'd probably want to do this only on the 1MB chunks jemalloc
> allocates for small and tiny allocations. For huge allocations (more
> than 1MB), it's likely the user will touch the whole thing, so huge
> pages could be a benefit.
>> jemalloc-discuss mailing list
>> jemalloc-discuss at canonware.com
More information about the jemalloc-discuss