Need help with patch allowing to iterate over all objects

[Benoit Jacob]

And another crash, also an assertion about nfree not having the expected value:

<jemalloc>: /hack/mozilla-central/memory/jemalloc/src/src/arena.c:165:
Failed assertion: "run->nfree < bin_info->nregs"

Program received signal SIGSEGV, Segmentation fault.
0x0000000000411605 in moz_abort () at
/hack/mozilla-central/memory/build/extraMallocFuncs.c:116
116       MOZ_CRASH();
(gdb) bt
#0  0x0000000000411605 in moz_abort () at
/hack/mozilla-central/memory/build/extraMallocFuncs.c:116
#1  0x000000000041743a in arena_run_reg_dalloc (run=0x7fffbe37f000,
ptr=0x7fffbe38af00) at
/hack/mozilla-central/memory/jemalloc/src/src/arena.c:165
#2  0x000000000041c5e7 in arena_dalloc_bin_locked
(arena=0x7ffff6c00180, chunk=0x7fffbe300000, ptr=0x7fffbe38af00,
mapelm=0x7fffbe300ce8) at
/hack/mozilla-central/memory/jemalloc/src/src/arena.c:1590
#3  0x000000000043c583 in tcache_bin_flush_small (tbin=0x7ffff6b022c8,
binind=21, rem=51, tcache=0x7ffff6b02000) at
/hack/mozilla-central/memory/jemalloc/src/src/tcache.c:128
#4  0x000000000043bdd3 in tcache_dalloc_small (tcache=0x7ffff6b02000,
ptr=0x7fffca24ff00, binind=21) at
/hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/tcache.h:399
#5  0x00000000004132e8 in arena_dalloc (arena=0x7ffff6c00180,
chunk=0x7fffca200000, ptr=0x7fffca24ff00, try_tcache=true) at
/hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/arena.h:956
#6  0x000000000042e13d in idalloc (ptr=0x7fffca24ff00) at
src/include/jemalloc/internal/jemalloc_internal.h:840
#7  0x000000000042e18f in iqalloc (ptr=0x7fffca24ff00) at
src/include/jemalloc/internal/jemalloc_internal.h:852
#8  0x0000000000432a58 in real_je_free (ptr=0x7fffca24ff00) at
/hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:1212
#9  0x00000000004330d8 in free (ptr=0x7fffca24ff20) at
/hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:1458
#10 0x00007ffff7fe8022 in moz_free (ptr=0x7fffca24ff20) at
/hack/mozilla-central/memory/mozalloc/mozalloc.cpp:51
#11 0x00007ffff472490c in nsStringBuffer::Release
(this=0x7fffca24ff20) at
/hack/mozilla-central/xpcom/string/src/nsSubstring.cpp:161
#12 0x00007ffff3e4c777 in FinalizeDOMString (fin=0x7ffff64b5058,
chars=0x7fffca24ff28) at
/hack/mozilla-central/js/xpconnect/src/XPCString.cpp:27
#13 0x00007ffff514a137 in JSExternalString::finalize
(this=0x7fffbf38d6c0, fop=0x7fffffffa000) at
/hack/mozilla-central/js/src/vm/String-inl.h:439
#14 0x00007ffff515f2a8 in js::gc::Arena::finalize<JSExternalString>
(this=0x7fffbf38d000, fop=0x7fffffffa000,
thingKind=js::gc::FINALIZE_EXTERNAL_STRING, thingSize=32) at
/hack/mozilla-central/js/src/jsgc.cpp:319


What can I do to help you help me?

Could anyone at least give me a quick explanation of what these
assertions are about?

Thanks!
Benoit


2012/8/21 Benoit Jacob <jacob.benoit.1 at gmail.com>:
> Hi,
>
> I am attaching a patch that allows to iterate over all objects.
>
> This is not a request for upstreaming: these patches cause crashes and I
> need help understanding these. And even if it didn't crash, this patch
> causes a severe overhead in memory usage and no attempt has been made to
> mitigate it. The goal of this effort is to allow custom developer builds of
> Firefox to introspect their own allocated blocks.
>
> The patch (attached to this email) works by renaming the public functions
> such as je_malloc() to real_je_malloc() and implementing custom je_malloc()
> to allocate a larger block, and use the extra space to store the data of a
> doubly-linked list element.
>
> In principle, this should be entirely transparent to the user (except for
> the increased memory usage), so I don't understand the crashes.
>
> The crashes aren't immediate: I can run Firefox (this patch if for Firefox's
> copy of jemalloc 3.0) for a while and browse a few pages without crashing.
>
> The crashes are assertion failures like this:
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x0000000000411605 in moz_abort ()
>     at /hack/mozilla-central/memory/build/extraMallocFuncs.c:116
> 116       MOZ_CRASH();
> (gdb) bt
> #0  0x0000000000411605 in moz_abort ()
>     at /hack/mozilla-central/memory/build/extraMallocFuncs.c:116
> #1  0x000000000041afdf in arena_bin_malloc_hard (arena=0x7ffff6c00180,
> bin=0x7ffff6c007c8)
>     at /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1189
> #2  0x000000000041b225 in arena_tcache_fill_small (arena=0x7ffff6c00180,
>     tbin=0x7ffff6b02148, binind=9, prof_accumbytes=0)
>     at /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1232
> #3  0x000000000043c239 in tcache_alloc_small_hard (tcache=0x7ffff6b02000,
>     tbin=0x7ffff6b02148, binind=9)
>     at /hack/mozilla-central/memory/jemalloc/src/src/tcache.c:72
> #4  0x000000000043b7dc in tcache_alloc_small (tcache=0x7ffff6b02000,
> size=160, zero=false)
>     at
> /hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/tcache.h:302
> #5  0x0000000000412c26 in arena_malloc (arena=0x0, size=160, zero=false,
> try_tcache=true)
>     at
> /hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/arena.h:869
> #6  0x000000000042dce8 in imalloc (size=160)
>     at src/include/jemalloc/internal/jemalloc_internal.h:735
> #7  0x000000000043111c in real_je_malloc (size=160)
>     at /hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:829
> #8  0x0000000000432fa3 in malloc (size=128)
>     at /hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:1425
> #9  0x00007ffff7fe803c in moz_xmalloc (size=128)
>     at /hack/mozilla-central/memory/mozalloc/mozalloc.cpp:57
> #10 0x00007ffff2dcb74d in nsTArrayInfallibleAllocator::Malloc (size=128)
>     at ../../dist/include/nsTArray.h:56
> ...
>
> Printing some variables here:
>
> (gdb) up
> #1  0x000000000041afdf in arena_bin_malloc_hard (arena=0x7ffff6c00180,
> bin=0x7ffff6c007c8)
>     at /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1189
> 1189            assert(bin->runcur->nfree > 0);
> (gdb) p bin
> $1 = (arena_bin_t *) 0x7ffff6c007c8
> (gdb) p *bin
> $2 = {lock = {lock = {__data = {__lock = 1, __count = 0, __owner = 6469,
> __nusers = 1,
>         __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
>       __size = "\001\000\000\000\000\000\000\000E\031\000\000\001", '\000'
> <repeats 26 times>, __align = 1}}, runcur = 0x7fffc2ced000, runs = {rbt_root
> = 0x7fffca3004d8, rbt_nil = {{
>         u = {rb_link = {rbn_left = 0x7ffff6c00800, rbn_right_red =
> 0x7ffff6c00800},
>           ql_link = {qre_next = 0x7ffff6c00800, qre_prev = 0x7ffff6c00800}},
>         prof_ctx = 0x7ffff6c00800}, bits = 0}}, stats = {allocated =
> 8896000,
>     nmalloc = 197757, ndalloc = 142157, nrequests = 300344, nfills = 5545,
>     nflushes = 2848, nruns = 2802, reruns = 5838, curruns = 1143}}
> (gdb) p bin->runcur
> $3 = (arena_run_t *) 0x7fffc2ced000
> (gdb) p *(bin->runcur)
> $4 = {bin = 0x7ffff6c007c8, nextind = 4544384, nfree = 0}
>
> Any help would be greatly appreciated.
>
> Thanks,
> Benoit