2MB (hugetlb) page support on Linux

[Vinay Y S]

Ok, I understand that due to more allocations per page the probability of a
page being released back to OS will reduce. But is it correct to say that
wastage will reduce for 4KB-8KB (and other higher) class allocations and
hence utilization will improve?

I also saw the 'medium' size class in the git log as well as on this
mailing list discussions. Is there a way to re-introduce that back with,
maybe, build-time configurable tradeoffs for memory efficiency vs (?)
lock-free purging?

Thanks,
Vinay

On Thu, Jan 31, 2013 at 5:16 AM, Jason Evans <jasone at canonware.com> wrote:

> On Jan 29, 2013, at 11:01 AM, Vinay Y S wrote:
> > I'm trying to use jemalloc as the allocator for Membase (a
> persistent/replicated memcached) on x86_64 Linux. With 4KB page size, the
> overhead (fragmentation) for allocations greater than 4KB (say 5000 bytes)
> that fall into 8KB 'large' allocation class is pretty high.
> >
> > To make the wastage a bit less, we would need more classes in the
> 'small' size. Given how page purge works, I understand just adding more
> classes in there or faking a higher page size won't work.
> >
> > Luckily my target OS - x86_64 linux supports 2MB huge pages. Can
> jemalloc take advantage of this? Is there a way to enable this?
>
> Huge pages actually make the dirty page purging problems worse with regard
> to fragmentation, because in order to keep huge pages intact, purging has
> to be all or nothing.  It is possible with a bit of trickery to get Linux
> to use huge pages for memory that jemalloc allocates, but it only makes
> sense to do so if the memory is densely used (lots of malloc()s, few
> free()s).
>
> Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20130131/5fb2e8e3/attachment.html>