Rounding up huge allocations to page boundaries instead of chunks

[Guilherme Goncalves]

----- Original Message -----
| From: "Jason Evans" <jasone at canonware.com>
| To: "Guilherme Goncalves" <ggp at mozilla.com>
| Cc: jemalloc-discuss at canonware.com
| Sent: Friday, September 5, 2014 6:40:08 PM
| Subject: Re: Rounding up huge allocations to page boundaries instead of chunks
| 
| We have strong pressure to actually map full chunks, so historically I held the
| opinion that if we're mapping the virtual memory, we might as well make it
| available to the application.

Fair enough, thanks for the explanation.

| Will this sufficiently address your accounting concerns?  There's the
| potential to over-report active memory by nearly 1.2X in the worst case, but
| that's a lot better than nearly 2X as things currently are.

While that's definitely better than 2X over-reporting, I wonder if we can't just expose the
sum of all huge allocations rounded to a page boundary as a new statistic, without actually
changing the way the mapping is done. That could give us the more accurate accounting we want
without causing fragmentation in the address space.

In more concrete terms, this would add a "stats.arenas.<i>.huge.allocated_pages" statistic,
reporting the total size of huge allocations serviced by the i-th arena, but rounded to pages
and not chunks (while still mapping memory in chunks as usual).

If I'm not missing anything, a patch to implement this would look similar yet a lot less
intrusive than my first attempt [1]. Does this sound reasonable?

Thanks!

1- https://github.com/guilherme-pg/jemalloc/commit/081feaed8b51deeb80c6d641745d8d0aefd6d883
-- 
Guilherme