[PATCH] Add a build option to freeze the number of arenas to 1

[Justin Lebar]

>> If any testing was done on Firefox with its fork of jemalloc to reach
>> this conclusion, surely it's no longer valid.
>>
>> I'd rather stick closer to the stock implementation until we can
>> demonstrate that changes are beneficial.
>
> I really don't think Firefox will benefit from having as much arenas as
> there are CPUs, and each thread being bound to a given arena.

Running stuff off main thread is not so uncommon these days.  Web
workers are the obvious case.  We intend to move image decoding off
main thread.  I recall that audio/video decoding may run off main
thread.  We intend to move compositing off main thread...

> I'm pretty certain this will significantly increase its memory footprint.

The overhead of an arena is

  constant bookkeeping (not affected by number of allocations) +
  proportional bookkeeping (roughly proportional to number of allocations) +
  fragmentation +
  pages which could be decommitted but haven't been yet

Proportional bookeeping is, in total, not affected, so we can ignore
it.  My guess is that constant bookkeeping is negligible.  I'd further
guess that adding arenas per-thread would likely decrease
fragmentation within each arena.  And jemalloc can bound the
not-yet-decommitted pages.

Anyway, I'm not saying that it's impossible or even unlikely that this
won't significantly increase memory usage, just that it's conceivable
that it might not be so bad.

> I also don't think things changed significantly with multiple arenas
> between the old jemalloc fork it uses and current jemalloc.

For one thing, would this change impact the lock-free alloc/dealloc
path, either in terms of correctness or performance?

But I also do not have evidence that one arena is the right thing for
Firefox even with the current jemalloc, so comparing then-and-now is
not particularly convincing...

> Anyways, besides Mozilla, I'm pretty sure this can have some other uses.

Perhaps, depending on the overhead.

I guess it's a relatively small change, so whatever.  I'm more
concerned with turning it on in Firefox without evidence that it's
beneficial.

-Justin