Live allocation iterator

[SNL]

Jason,

One more query on same line.

Assume that base and extent are start and size of allocation. Is there
infrastructure in Jemalloc to lookup an address (base + n ), n < extent ?
My understanding is that such infrastructure does not exist in jemalloc. My
use case involves checking whether a given pointer is part of a valid
allocation or not.



On Wed, Mar 25, 2015 at 2:14 PM, SNL <snl20465 at gmail.com> wrote:

> This looks relevant but never was pushed upstream.
>
> https://www.mail-archive.com/jemalloc-discuss@canonware.com/msg00027.html
>
>
> Any thoughts on this patch set keeping in mind jemalloc-dev current state
> ? Any design/architectural inputs are also appreciated. This seems to have
> worked well for Alessandro who shared the patchset, I dont see these
> changes in their current code base though.
>
> On Wed, Mar 25, 2015 at 11:25 AM, SNL <snl20465 at gmail.com> wrote:
>
>> Does your use case allow you to wrap the allocator and keep a side table
>> which supports iteration, or can you trace allocation activity and
>> post-process it?
>>
>> => This is what I have implemented currently but this does not scale for
>> multithreaded programs due to locking issues. I was pondering over idea of
>> not doing any book keeping myself and use jemalloc data structures ( and
>> efficient low level locking ) for my purpose.
>>
>> Is this not possible even when all my metadata objects ( which hold user
>> allocation base and extent ) are fixed sized and allocated from a separate
>> arena ? I was under the impression that I should be able to mask and find
>> out offsets within chunk->region->pages and find out objects. As my objects
>> are all fixed size, they would be contiguous in memory. Is this valid
>> understanding ?
>>
>>
>> On Tue, Mar 24, 2015 at 11:16 PM, Jason Evans <jasone at canonware.com>
>> wrote:
>>
>>> On Mar 24, 2015, at 6:53 AM, SNL <snl20465 at gmail.com> wrote:
>>>
>>> If this is not possible ( I see an old post which says jemalloc does not
>>> have this infra but that was back in 2012 ). Is it possible to iterate over
>>> just an arena and dump all allocations in it  ? Basically, I have memory
>>> pool implemented on top of jemalloc which holds metadata about all other
>>> allocations jemalloc does, if I can iterate over this arena and get hold of
>>> metadata, I can get hold of each allocation as well.
>>>
>>> Any thoughts would be appreciated.
>>>
>>> On Tue, Mar 24, 2015 at 5:28 PM, SNL <snl20465 at gmail.com> wrote:
>>>
>>>>
>>>> My use case is to walk through all live allocations ( the ones which
>>>> are not freed yet) of all sizes the program ever made and dump it as the
>>>> end of the program or on demand. I have looked at stats but I want
>>>> something with even more details. Basically, the function should be able to
>>>> iterate through all arenas and print details on following lines:
>>>>
>>>> arena = 1 base = 0x7fd7beb88800 size = 15 bytes etc
>>>>
>>>> Is there any api in jemalloc which already does subset of this  ? Any
>>>> pointers will be helpful.
>>>>
>>>
>>> jemalloc doesn't support iteration over live allocations, and it's
>>> unlikely to ever directly support iteration, because the feature imposes
>>> nontrivial additional complexity for limited benefit.  The closest thing
>>> under consideration is https://github.com/jemalloc/jemalloc/issues/203,
>>> but I don't plan to work on that until at least next year.
>>>
>>> Does your use case allow you to wrap the allocator and keep a side table
>>> which supports iteration, or can you trace allocation activity and
>>> post-process it?
>>>
>>> Jason
>>>
>>
>>
>>
>> --
>>
>> Cheers,
>> Sunny.
>>
>
>
>
> --
>
> Cheers,
> Sunny.
>



-- 

Cheers,
Sunny.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20150420/13c588b2/attachment.html>