Patch: enumerate all live blocks

[Salvatore Sanfilippo]

Hello,

what is the advantage of this approach compared to doing it entirely
in the application code just wrapping malloc/realloc/free?
Basically wrappered_malloc() allocates a bit more space accordingly to
the metadata to store, store this metadata at the start of the block,
and then returns the pointer advanced to the start of the empty space.

Doing it in the context of the application makes it
malloc-implementation agnostic that can be an advantage.

Regards,
Salvatore

On Sat, Oct 6, 2012 at 7:28 AM, Benoit Jacob <jacob.benoit.1 at gmail.com> wrote:
> Hello,
>
> The attached patch instruments jemalloc 3.0, adding the ability to
> enumerate all live blocks.
>
> Currently, the only information stored about blocks is their (payload)
> address and size, but the plan is to also store their allocation call
> stack.
>
> This is achieved by allocating larger blocks than requested, and using
> the extra space to store doubly linked list elements.
>
>  This is provided just in case it might be useful to anyone, not
> considered ready for inclusion in jemalloc. It's been tested for 15
> minutes in a Firefox build.
>
> Details about the overhead, and how it could be reduced:
>  * Memory overhead is, per block: 32 bytes on 32-bit systems, 48 bytes
> on 64-bit systems (assuming size_t == uintptr_t). Could easily be
> reduced to 16 bytes in both cases (by using a XOR linked list and
> assuming that no block exceeds 4G).
>  * Slowness overhead is essentially an additional mutex to lock on
> every malloc/free call. Could be solved in various ways, either
> copying what jemalloc does internally, or by using a lock-free list.
>
> If you want to test it out, currently the only built-in way to output
> the list of blocks is to call je_dump_list_of_blocks(void) e.g. from
> your debugger. See its code to see the relevant calls. Sample output
> from Firefox:
>
> ...snip...
> Block 193965:  real block = 0x7fffba218580,  payload = 0x7fffba2185b0,
>  payload size = 64
> Block 193966:  real block = 0x7fffc02dd000,  payload = 0x7fffc02dd030,
>  payload size = 1024
> Block 193967:  real block = 0x7fffc2053ce0,  payload = 0x7fffc2053d10,
>  payload size = 24
> Block 193968:  real block = 0x7fffc5b4ed80,  payload = 0x7fffc5b4edb0,
>  payload size = 80
> Block 193969:  real block = 0x7fffd119e240,  payload = 0x7fffd119e270,
>  payload size = 64
> Block 193970:  real block = 0x7fffcfb4aa60,  payload = 0x7fffcfb4aa90,
>  payload size = 24
> Block 193971:  real block = 0x7fffbb85e1f0,  payload = 0x7fffbb85e220,
>  payload size = 24
> Block 193972:  real block = 0x7fffe20a62e0,  payload = 0x7fffe20a6310,
>  payload size = 32
>
> End enumeration of 193973 jemalloc blocks.
>
> Cheers,
> Benoit
>
> _______________________________________________
> jemalloc-discuss mailing list
> jemalloc-discuss at canonware.com
> http://www.canonware.com/mailman/listinfo/jemalloc-discuss
>



-- 
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org

Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
       — David Gelernter