Live allocation iterator

[SNL]

Does your use case allow you to wrap the allocator and keep a side table
which supports iteration, or can you trace allocation activity and
post-process it?

=> This is what I have implemented currently but this does not scale for
multithreaded programs due to locking issues. I was pondering over idea of
not doing any book keeping myself and use jemalloc data structures ( and
efficient low level locking ) for my purpose.

Is this not possible even when all my metadata objects ( which hold user
allocation base and extent ) are fixed sized and allocated from a separate
arena ? I was under the impression that I should be able to mask and find
out offsets within chunk->region->pages and find out objects. As my objects
are all fixed size, they would be contiguous in memory. Is this valid
understanding ?


On Tue, Mar 24, 2015 at 11:16 PM, Jason Evans <jasone at canonware.com> wrote:

> On Mar 24, 2015, at 6:53 AM, SNL <snl20465 at gmail.com> wrote:
>
> If this is not possible ( I see an old post which says jemalloc does not
> have this infra but that was back in 2012 ). Is it possible to iterate over
> just an arena and dump all allocations in it  ? Basically, I have memory
> pool implemented on top of jemalloc which holds metadata about all other
> allocations jemalloc does, if I can iterate over this arena and get hold of
> metadata, I can get hold of each allocation as well.
>
> Any thoughts would be appreciated.
>
> On Tue, Mar 24, 2015 at 5:28 PM, SNL <snl20465 at gmail.com> wrote:
>
>>
>> My use case is to walk through all live allocations ( the ones which are
>> not freed yet) of all sizes the program ever made and dump it as the end of
>> the program or on demand. I have looked at stats but I want something with
>> even more details. Basically, the function should be able to iterate
>> through all arenas and print details on following lines:
>>
>> arena = 1 base = 0x7fd7beb88800 size = 15 bytes etc
>>
>> Is there any api in jemalloc which already does subset of this  ? Any
>> pointers will be helpful.
>>
>
> jemalloc doesn't support iteration over live allocations, and it's
> unlikely to ever directly support iteration, because the feature imposes
> nontrivial additional complexity for limited benefit.  The closest thing
> under consideration is https://github.com/jemalloc/jemalloc/issues/203,
> but I don't plan to work on that until at least next year.
>
> Does your use case allow you to wrap the allocator and keep a side table
> which supports iteration, or can you trace allocation activity and
> post-process it?
>
> Jason
>



-- 

Cheers,
Sunny.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20150325/6fb1810b/attachment.html>