keeping memory usage at certain limit

[Antony Dovgal]

On 04/29/2014 03:43 PM, Ricardo Nabinger Sanchez wrote:
> On Mon, 28 Apr 2014 15:08:16 +0400
> Antony Dovgal <antony.dovgal at gmail.com> wrote:
>
>> I suspect that this is caused by memory fragmentation issues, though I've no idea how to prove it (or at least all my ideas involve complex stats and are quite inefficient).
>
> You could dump your pointers and their sizes, and plot that information
> graphically, including relevant info from jemalloc (map bases and their
> sizes as well).  If it's heavily fragmented, it will be apparent.

Dumping millions of pointers and hashtables (I use Judy arrays a lot) to plot the data - that's what I meant by complex stats, yes =).

> If that is indeed your case, and you can modify your data container,
> perhaps switching to index-based access could solve your problem.  That
> would require you to forget about pointers and instead use an index, so
> that your data container is free to move data around and shrink the
> memory in use.  There is a lot of management and memory copying,
> meaning the impact of such a change should be quite perceptible.
>
> Of course, this is no trivial change.  Dealing with long- and
> short-lived objects is challenging.  :-)

Yeah, it's quite non-trivial.
Thanks for the suggestion, though, I'll keep it in mind.

-- 
Wbr,
Antony Dovgal