Memory usage regression

[Mike Hommey]

On Tue, Oct 30, 2012 at 04:35:53PM -0700, Jason Evans wrote:
> On Oct 30, 2012, at 12:49 PM, Jason Evans wrote:
> > The preference for allocating dirty runs was a solution to excessive
> > dirty page purging.  However, the purging policy (as of jemalloc
> > 3.0.0) is round-robin, justified only as a strategy for allowing
> > dirty pages to accumulate in chunks before going to the considerable
> > effort (including arena mutex operations) of scanning a chunk for
> > dirty pages.  In retrospect I'm thinking maybe this was a bad
> > choice, and that we should go back to scanning downward through
> > memory to purge dirty pages.  The danger is that the linear scanning
> > overhead for scanning each chunk will cause a measurable performance
> > degradation if high chunks routinely have many runs, only a few of
> > which are unused dirty runs.  I think that problem can be solved
> > with slightly more sophisticated hysteresis though.
> > 
> > I'll work on a diff for you to test, and see how it affects Firefox.
> > I'll do some testing with Facebook server loads too (quite different
> > behavior from Firefox).  If this causes a major reduction in virtual
> > memory usage for both workloads, it's probably the right thing to
> > do, even speed-wise.
> 
> Here's a very lightly tested patch.  Apologies if it's buggy, but I'm
> out of time for today.

It's unfortunately only slightly better.
http://i.imgur.com/hN1Cj.png

Mike