2MB (hugetlb) page support on Linux

[Jason Evans]

On Jan 29, 2013, at 11:01 AM, Vinay Y S wrote:
> I'm trying to use jemalloc as the allocator for Membase (a persistent/replicated memcached) on x86_64 Linux. With 4KB page size, the overhead (fragmentation) for allocations greater than 4KB (say 5000 bytes) that fall into 8KB 'large' allocation class is pretty high.
> 
> To make the wastage a bit less, we would need more classes in the 'small' size. Given how page purge works, I understand just adding more classes in there or faking a higher page size won't work.
> 
> Luckily my target OS - x86_64 linux supports 2MB huge pages. Can jemalloc take advantage of this? Is there a way to enable this?

Huge pages actually make the dirty page purging problems worse with regard to fragmentation, because in order to keep huge pages intact, purging has to be all or nothing.  It is possible with a bit of trickery to get Linux to use huge pages for memory that jemalloc allocates, but it only makes sense to do so if the memory is densely used (lots of malloc()s, few free()s).

Jason