keeping memory usage at certain limit
antony.dovgal at gmail.com
Tue Apr 29 06:00:24 PDT 2014
On 04/29/2014 03:43 PM, Ricardo Nabinger Sanchez wrote:
> On Mon, 28 Apr 2014 15:08:16 +0400
> Antony Dovgal <antony.dovgal at gmail.com> wrote:
>> I suspect that this is caused by memory fragmentation issues, though I've no idea how to prove it (or at least all my ideas involve complex stats and are quite inefficient).
> You could dump your pointers and their sizes, and plot that information
> graphically, including relevant info from jemalloc (map bases and their
> sizes as well). If it's heavily fragmented, it will be apparent.
Dumping millions of pointers and hashtables (I use Judy arrays a lot) to plot the data - that's what I meant by complex stats, yes =).
> If that is indeed your case, and you can modify your data container,
> perhaps switching to index-based access could solve your problem. That
> would require you to forget about pointers and instead use an index, so
> that your data container is free to move data around and shrink the
> memory in use. There is a lot of management and memory copying,
> meaning the impact of such a change should be quite perceptible.
> Of course, this is no trivial change. Dealing with long- and
> short-lived objects is challenging. :-)
Yeah, it's quite non-trivial.
Thanks for the suggestion, though, I'll keep it in mind.
More information about the jemalloc-discuss