jemalloc 3 performance vs. mozjemalloc

[Jason Evans]

On Feb 3, 2015, at 4:40 PM, Mike Hommey <mh at glandium.org> wrote:
> On Tue, Feb 03, 2015 at 04:19:00PM -0800, Jason Evans wrote:
>> On Feb 3, 2015, at 2:51 PM, Mike Hommey <mh at glandium.org> wrote:
>>> I've been tracking a startup time regression in Firefox for Android
>>> when we tried to switch from mozjemalloc (memory refresher: it's
>>> derived from jemalloc 0.9) to mostly current jemalloc dev.
>>> 
>>> It turned out to be https://github.com/jemalloc/jemalloc/pull/192
>> 
>> I intentionally removed the functionality #192 adds back (in
>> e3d13060c8a04f08764b16b003169eb205fa09eb), but apparently forgot to
>> update the documentation.  Do you have an understanding of why it's
>> hurting performance so much?
> 
> My understanding is that the huge increase in page faults is making the
> difference. On Firefox startup we go from 50k page faults to 35k with
> that patch. I can surely double check whether it's really the page
> faults, or if it's actually the madvising itself that causes the
> regression. Or both.
> 
>> Is there any chance you can make your test case available so I can dig
>> in further?
> 
> https://gist.githubusercontent.com/glandium/a42d0265e324688cafc4/raw/gistfile1.c

I added some logging and determined that ~90% of the dirty page purging is happening in the first 2% of the allocation trace.  This appears to be almost entirely due to repeated 32 KiB allocation/deallocation.

I still have vague plans to add time-based hysteresis mechanisms so that #192 isn't necessary, but until then, #192 it is.

Thank you for the detailed feedback.  It's scary to hear of such performance regressions without a way to gain insight into what's going on.

Jason