Transparent Huge Pages

[Justin Lebar]

Okay, this behavior is not entirely ridiculous, but at least Firefox's
fork of jemalloc will need to change to work well with this.

What happens if you MADV_DONTNEED all but the first 4k after you touch
the first byte?  What about if you MADV_DONTNEED the whole thing
before you touch any part?

On Mon, Feb 20, 2012 at 7:55 PM, Jakob Blomer <jakob.blomer at cern.ch> wrote:
> After thinking a bit more about it, I don't think it's a bug but this is
> just the way transparent huge pages work.  For properly aligned memory, the
> kernel takes a 2M page.  This just means 2M of real memory are gone, and I
> think not even splitting afterwards can change that.
>
> The following program requires 300-400k RSS without transparent huge pages,
> but >2M with THP.
>
> #include <unistd.h>
> #include <sys/mman.h>
> #include <stdio.h>
> #include <errno.h>
>
> int main() {
>  int size = 4*1024*1024;
>  int _2m = 2*1024*1024;
>  char *mapping = mmap(0x42000000, size, PROT_READ | PROT_WRITE,
>                       MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
>  mapping[0] = '\0';
>  printf("Region of size %d mapped at %p (error %d), aligned at 2M: %d\n",
> size, mapping, errno, (long)mapping%_2m);
>
>  sleep(30);
>  return 0;
> }
>
> Cheers,
> Jakob
>
>
> On 2/20/12 5:10 PM, Justin Lebar wrote:
>>
>> Hm, upon further consideration...
>>
>> If you mmap a huge page (say, 1MB), then MADV_DONTNEED a few 4-KB
>> chunks inside, transparent huge pages should break up the huge page so
>> it can decommit the parts I asked it to decommit.  If it doesn't, that
>> sounds like a kernel bug to me!
>>
>> Similarly, if I mmap 1MB, get a huge page, and then touch only a few
>> bytes in the middle, the kernel shouldn't commit a huge page.
>>
>> If huge pages is behaving how I expect, I don't see why it would cause
>> your application to use more memory.
>>
>> Just to check, you're measuring RSS, not vsize, right?
>>
>> On Mon, Feb 20, 2012 at 4:59 PM, Justin Lebar<justin.lebar at gmail.com>
>>  wrote:
>>>>
>>>> jemalloc seems to be prone to transparent huge pages
>>>> (https://lwn.net/Articles/423584), presumably due to its use of mmap().
>>>>  In
>>>> my case (fuse module), the initial memory consumption jumped from ~12M
>>>> to
>>>> ~27M.  The use of --enable-dss helps a little, bringing the consumption
>>>> down
>>>> to ~19M.
>>>
>>>
>>> Ouch!
>>>
>>>> Did anyone else experienced similar behavior?  Is there an easy way of
>>>> avoiding transparent huge pages for jemalloc'ed memory?  The only
>>>> workaround
>>>> that comes to my mind is a malloc wrapper that runs madvise(...,
>>>> MADV_NOHUGEPAGE) on every newly allocated chunk.
>>>
>>>
>>> You'd probably want to do this only on the 1MB chunks jemalloc
>>> allocates for small and tiny allocations.  For huge allocations (more
>>> than 1MB), it's likely the user will touch the whole thing, so huge
>>> pages could be a benefit.
>>>
>>>>
>>>> Cheers,
>>>> Jakob
>>>>
>>>> _______________________________________________
>>>> jemalloc-discuss mailing list
>>>> jemalloc-discuss at canonware.com
>>>> http://www.canonware.com/mailman/listinfo/jemalloc-discuss
>>
>> .
>>
>