Jemalloc stats

[Jason Evans]

On Mar 22, 2012, at 11:03 AM, Mike Hommey wrote:
> In Firefox, we're tracking some of the stats provided by our fork of
> jemalloc. One of them is:
> - HeapCommitted - Memory mapped by the heap allocator that is committed,
>  i.e. in physical memory or paged to disk.  When heap-committed is
>  larger than heap-allocated, the difference between the two values is
>  likely due to external fragmentation; that is, the allocator allocated
>  a large block of memory and is unable to decommit it because a small
>  part of that block is currently in use.
> 
> It would seem like this could match stats.active, but i'm not entirely
> sure it is the case. In particular, we don't count madvised pages in
> that metric, but it would seem stats.active does, although I haven't dug
> deep enough yet.

stats.active tracks all pages with active application allocations in them.  It does not include dirty unused pages for which madvise() has not yet been called, nor does it include pages that are entirely devoted to allocator metadata.  stats.allocated/stats.active is a precise indicator for external fragmentation, but since stats.active does not incorporate allocator metadata (e.g., chunk header pages), it does not represent the complete size of "committed" memory.  It would be possible to track the metadata pages with little extra effort, but I haven't found it to be very important in understanding what's going on internally, so I haven't added it.  Is the distinction important for Firefox?

Thanks,
Jason