Need help with patch allowing to iterate over all objects

[Benoit Jacob]

Hi,

I am attaching a patch that allows to iterate over all objects.

This is not a request for upstreaming: these patches cause crashes and I
need help understanding these. And even if it didn't crash, this patch
causes a severe overhead in memory usage and no attempt has been made to
mitigate it. The goal of this effort is to allow custom developer builds of
Firefox to introspect their own allocated blocks.

The patch (attached to this email) works by renaming the public functions
such as je_malloc() to real_je_malloc() and implementing custom je_malloc()
to allocate a larger block, and use the extra space to store the data of a
doubly-linked list element.

In principle, this should be entirely transparent to the user (except for
the increased memory usage), so I don't understand the crashes.

The crashes aren't immediate: I can run Firefox (this patch if for
Firefox's copy of jemalloc 3.0) for a while and browse a few pages without
crashing.

The crashes are assertion failures like this:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000411605 in moz_abort ()
    at /hack/mozilla-central/memory/build/extraMallocFuncs.c:116
116       MOZ_CRASH();
(gdb) bt
#0  0x0000000000411605 in moz_abort ()
    at /hack/mozilla-central/memory/build/extraMallocFuncs.c:116
#1  0x000000000041afdf in arena_bin_malloc_hard (arena=0x7ffff6c00180,
bin=0x7ffff6c007c8)
    at /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1189
#2  0x000000000041b225 in arena_tcache_fill_small (arena=0x7ffff6c00180,
    tbin=0x7ffff6b02148, binind=9, prof_accumbytes=0)
    at /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1232
#3  0x000000000043c239 in tcache_alloc_small_hard (tcache=0x7ffff6b02000,
    tbin=0x7ffff6b02148, binind=9)
    at /hack/mozilla-central/memory/jemalloc/src/src/tcache.c:72
#4  0x000000000043b7dc in tcache_alloc_small (tcache=0x7ffff6b02000,
size=160, zero=false)
    at
/hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/tcache.h:302
#5  0x0000000000412c26 in arena_malloc (arena=0x0, size=160, zero=false,
try_tcache=true)
    at
/hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/arena.h:869
#6  0x000000000042dce8 in imalloc (size=160)
    at src/include/jemalloc/internal/jemalloc_internal.h:735
#7  0x000000000043111c in real_je_malloc (size=160)
    at /hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:829
#8  0x0000000000432fa3 in malloc (size=128)
    at /hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:1425
#9  0x00007ffff7fe803c in moz_xmalloc (size=128)
    at /hack/mozilla-central/memory/mozalloc/mozalloc.cpp:57
#10 0x00007ffff2dcb74d in nsTArrayInfallibleAllocator::Malloc (size=128)
    at ../../dist/include/nsTArray.h:56
...

Printing some variables here:

(gdb) up
#1  0x000000000041afdf in arena_bin_malloc_hard (arena=0x7ffff6c00180,
bin=0x7ffff6c007c8)
    at /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1189
1189            assert(bin->runcur->nfree > 0);
(gdb) p bin
$1 = (arena_bin_t *) 0x7ffff6c007c8
(gdb) p *bin
$2 = {lock = {lock = {__data = {__lock = 1, __count = 0, __owner = 6469,
__nusers = 1,
        __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
      __size = "\001\000\000\000\000\000\000\000E\031\000\000\001", '\000'
<repeats 26 times>, __align = 1}}, runcur = 0x7fffc2ced000, runs =
{rbt_root = 0x7fffca3004d8, rbt_nil = {{
        u = {rb_link = {rbn_left = 0x7ffff6c00800, rbn_right_red =
0x7ffff6c00800},
          ql_link = {qre_next = 0x7ffff6c00800, qre_prev =
0x7ffff6c00800}},
        prof_ctx = 0x7ffff6c00800}, bits = 0}}, stats = {allocated =
8896000,
    nmalloc = 197757, ndalloc = 142157, nrequests = 300344, nfills = 5545,
    nflushes = 2848, nruns = 2802, reruns = 5838, curruns = 1143}}
(gdb) p bin->runcur
$3 = (arena_run_t *) 0x7fffc2ced000
(gdb) p *(bin->runcur)
$4 = {bin = 0x7ffff6c007c8, nextind = 4544384, nfree = 0}

Any help would be greatly appreciated.

Thanks,
Benoit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20120821/79d730e9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jemalloc-instrument
Type: application/octet-stream
Size: 14337 bytes
Desc: not available
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20120821/79d730e9/attachment.obj>