More aggressive page purging
jasone at canonware.com
Fri May 11 16:16:16 PDT 2012
On May 10, 2012, at 11:55 AM, Justin Lebar wrote:
>>> In summary, I don't think there's a problem here to fix. Am I missing
>> We could use RSS - number of madvised pages, for sure, but that doesn't
>> quite help with people looking at their task manager ans seeing memory
>> usage 1GB higher than what it actually is. (And it doesn't help to make
>> the numbers Firefox itself reports believable)
> Yeah, the perception problem ("I looked at the task manager and
> Firefox sux!") is a real one that, from MemShrink's perspective, we
> need to solve.
It's unfortunate that the operating systems don't expose statistics that provide clarity here. I don't want to make aggressive purging default behavior, but I'm open to patches that make this a configuration-time behavior after the 3.0.0 release (probably today or tomorrow).
Re: stealing a bit for 'decommit', overloading the 'large' bit is probably the easiest approach. The recent arena_mapbits_*() refactoring should make decommit support much easier to add.
> But also, |RSS minus madvised memory| is wrong as soon as any of the
> madvised memory is kicked out due to memory pressure. So without
> double-purge semantics, I don't see how you'd be able to reliably
> measure "hard" memory usage.
jemalloc maintains 'allocated', 'active', and 'mapped' statistics. 'active' ignores a bit of internally allocated metadata memory, but it is very close to being RSS minus madvise'd memory minus non-malloc memory (libraries, other mapped files, etc.).
More information about the jemalloc-discuss