Transparent Huge Pages

[Jakob Blomer]

After thinking a bit more about it, I don't think it's a bug but this is 
just the way transparent huge pages work.  For properly aligned memory, 
the kernel takes a 2M page.  This just means 2M of real memory are gone, 
and I think not even splitting afterwards can change that.

The following program requires 300-400k RSS without transparent huge 
pages, but >2M with THP.

#include <unistd.h>
#include <sys/mman.h>
#include <stdio.h>
#include <errno.h>

int main() {
   int size = 4*1024*1024;
   int _2m = 2*1024*1024;
   char *mapping = mmap(0x42000000, size, PROT_READ | PROT_WRITE,
                        MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
   mapping[0] = '\0';
   printf("Region of size %d mapped at %p (error %d), aligned at 2M: 
%d\n", size, mapping, errno, (long)mapping%_2m);

   sleep(30);
   return 0;
}

Cheers,
Jakob

On 2/20/12 5:10 PM, Justin Lebar wrote:
> Hm, upon further consideration...
>
> If you mmap a huge page (say, 1MB), then MADV_DONTNEED a few 4-KB
> chunks inside, transparent huge pages should break up the huge page so
> it can decommit the parts I asked it to decommit.  If it doesn't, that
> sounds like a kernel bug to me!
>
> Similarly, if I mmap 1MB, get a huge page, and then touch only a few
> bytes in the middle, the kernel shouldn't commit a huge page.
>
> If huge pages is behaving how I expect, I don't see why it would cause
> your application to use more memory.
>
> Just to check, you're measuring RSS, not vsize, right?
>
> On Mon, Feb 20, 2012 at 4:59 PM, Justin Lebar<justin.lebar at gmail.com>  wrote:
>>> jemalloc seems to be prone to transparent huge pages
>>> (https://lwn.net/Articles/423584), presumably due to its use of mmap().  In
>>> my case (fuse module), the initial memory consumption jumped from ~12M to
>>> ~27M.  The use of --enable-dss helps a little, bringing the consumption down
>>> to ~19M.
>>
>> Ouch!
>>
>>> Did anyone else experienced similar behavior?  Is there an easy way of
>>> avoiding transparent huge pages for jemalloc'ed memory?  The only workaround
>>> that comes to my mind is a malloc wrapper that runs madvise(...,
>>> MADV_NOHUGEPAGE) on every newly allocated chunk.
>>
>> You'd probably want to do this only on the 1MB chunks jemalloc
>> allocates for small and tiny allocations.  For huge allocations (more
>> than 1MB), it's likely the user will touch the whole thing, so huge
>> pages could be a benefit.
>>
>>>
>>> Cheers,
>>> Jakob
>>>
>>> _______________________________________________
>>> jemalloc-discuss mailing list
>>> jemalloc-discuss at canonware.com
>>> http://www.canonware.com/mailman/listinfo/jemalloc-discuss
> .
>