Need help with patch allowing to iterate over all objects

Fri Aug 24 15:26:44 PDT 2012

My guess is that the application is passing a bogus pointer to free(), perhaps one that had already been freed.

Jason

On Aug 24, 2012, at 12:53 PM, Benoit Jacob wrote:
> In the assertion failure in my last email, I have these local variables:
> 
> (gdb) p run->nfree
> $1 = 32767
> (gdb) p bin_info->nregs
> $2 = 51
> 
> Benoit
> 
> 2012/8/24 Benoit Jacob <jacob.benoit.1 at gmail.com>:
>> And another crash, also an assertion about nfree not having the expected value:
>> 
>> <jemalloc>: /hack/mozilla-central/memory/jemalloc/src/src/arena.c:165:
>> Failed assertion: "run->nfree < bin_info->nregs"
>> 
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x0000000000411605 in moz_abort () at
>> /hack/mozilla-central/memory/build/extraMallocFuncs.c:116
>> 116       MOZ_CRASH();
>> (gdb) bt
>> #0  0x0000000000411605 in moz_abort () at
>> /hack/mozilla-central/memory/build/extraMallocFuncs.c:116
>> #1  0x000000000041743a in arena_run_reg_dalloc (run=0x7fffbe37f000,
>> ptr=0x7fffbe38af00) at
>> /hack/mozilla-central/memory/jemalloc/src/src/arena.c:165
>> #2  0x000000000041c5e7 in arena_dalloc_bin_locked
>> (arena=0x7ffff6c00180, chunk=0x7fffbe300000, ptr=0x7fffbe38af00,
>> mapelm=0x7fffbe300ce8) at
>> /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1590
>> #3  0x000000000043c583 in tcache_bin_flush_small (tbin=0x7ffff6b022c8,
>> binind=21, rem=51, tcache=0x7ffff6b02000) at
>> /hack/mozilla-central/memory/jemalloc/src/src/tcache.c:128
>> #4  0x000000000043bdd3 in tcache_dalloc_small (tcache=0x7ffff6b02000,
>> ptr=0x7fffca24ff00, binind=21) at
>> /hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/tcache.h:399
>> #5  0x00000000004132e8 in arena_dalloc (arena=0x7ffff6c00180,
>> chunk=0x7fffca200000, ptr=0x7fffca24ff00, try_tcache=true) at
>> /hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/arena.h:956
>> #6  0x000000000042e13d in idalloc (ptr=0x7fffca24ff00) at
>> src/include/jemalloc/internal/jemalloc_internal.h:840
>> #7  0x000000000042e18f in iqalloc (ptr=0x7fffca24ff00) at
>> src/include/jemalloc/internal/jemalloc_internal.h:852
>> #8  0x0000000000432a58 in real_je_free (ptr=0x7fffca24ff00) at
>> /hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:1212
>> #9  0x00000000004330d8 in free (ptr=0x7fffca24ff20) at
>> /hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:1458
>> #10 0x00007ffff7fe8022 in moz_free (ptr=0x7fffca24ff20) at
>> /hack/mozilla-central/memory/mozalloc/mozalloc.cpp:51
>> #11 0x00007ffff472490c in nsStringBuffer::Release
>> (this=0x7fffca24ff20) at
>> /hack/mozilla-central/xpcom/string/src/nsSubstring.cpp:161
>> #12 0x00007ffff3e4c777 in FinalizeDOMString (fin=0x7ffff64b5058,
>> chars=0x7fffca24ff28) at
>> /hack/mozilla-central/js/xpconnect/src/XPCString.cpp:27
>> #13 0x00007ffff514a137 in JSExternalString::finalize
>> (this=0x7fffbf38d6c0, fop=0x7fffffffa000) at
>> /hack/mozilla-central/js/src/vm/String-inl.h:439
>> #14 0x00007ffff515f2a8 in js::gc::Arena::finalize<JSExternalString>
>> (this=0x7fffbf38d000, fop=0x7fffffffa000,
>> thingKind=js::gc::FINALIZE_EXTERNAL_STRING, thingSize=32) at
>> /hack/mozilla-central/js/src/jsgc.cpp:319
>> 
>> 
>> What can I do to help you help me?
>> 
>> Could anyone at least give me a quick explanation of what these
>> assertions are about?
>> 
>> Thanks!
>> Benoit
>> 
>> 
>> 2012/8/21 Benoit Jacob <jacob.benoit.1 at gmail.com>:
>>> Hi,
>>> 
>>> I am attaching a patch that allows to iterate over all objects.
>>> 
>>> This is not a request for upstreaming: these patches cause crashes and I
>>> need help understanding these. And even if it didn't crash, this patch
>>> causes a severe overhead in memory usage and no attempt has been made to
>>> mitigate it. The goal of this effort is to allow custom developer builds of
>>> Firefox to introspect their own allocated blocks.
>>> 
>>> The patch (attached to this email) works by renaming the public functions
>>> such as je_malloc() to real_je_malloc() and implementing custom je_malloc()
>>> to allocate a larger block, and use the extra space to store the data of a
>>> doubly-linked list element.
>>> 
>>> In principle, this should be entirely transparent to the user (except for
>>> the increased memory usage), so I don't understand the crashes.
>>> 
>>> The crashes aren't immediate: I can run Firefox (this patch if for Firefox's
>>> copy of jemalloc 3.0) for a while and browse a few pages without crashing.
>>> 
>>> The crashes are assertion failures like this:
>>> 
>>> Program received signal SIGSEGV, Segmentation fault.
>>> 0x0000000000411605 in moz_abort ()
>>>    at /hack/mozilla-central/memory/build/extraMallocFuncs.c:116
>>> 116       MOZ_CRASH();
>>> (gdb) bt
>>> #0  0x0000000000411605 in moz_abort ()
>>>    at /hack/mozilla-central/memory/build/extraMallocFuncs.c:116
>>> #1  0x000000000041afdf in arena_bin_malloc_hard (arena=0x7ffff6c00180,
>>> bin=0x7ffff6c007c8)
>>>    at /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1189
>>> #2  0x000000000041b225 in arena_tcache_fill_small (arena=0x7ffff6c00180,
>>>    tbin=0x7ffff6b02148, binind=9, prof_accumbytes=0)
>>>    at /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1232
>>> #3  0x000000000043c239 in tcache_alloc_small_hard (tcache=0x7ffff6b02000,
>>>    tbin=0x7ffff6b02148, binind=9)
>>>    at /hack/mozilla-central/memory/jemalloc/src/src/tcache.c:72
>>> #4  0x000000000043b7dc in tcache_alloc_small (tcache=0x7ffff6b02000,
>>> size=160, zero=false)
>>>    at
>>> /hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/tcache.h:302
>>> #5  0x0000000000412c26 in arena_malloc (arena=0x0, size=160, zero=false,
>>> try_tcache=true)
>>>    at
>>> /hack/mozilla-central/memory/jemalloc/src/include/jemalloc/internal/arena.h:869
>>> #6  0x000000000042dce8 in imalloc (size=160)
>>>    at src/include/jemalloc/internal/jemalloc_internal.h:735
>>> #7  0x000000000043111c in real_je_malloc (size=160)
>>>    at /hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:829
>>> #8  0x0000000000432fa3 in malloc (size=128)
>>>    at /hack/mozilla-central/memory/jemalloc/src/src/jemalloc.c:1425
>>> #9  0x00007ffff7fe803c in moz_xmalloc (size=128)
>>>    at /hack/mozilla-central/memory/mozalloc/mozalloc.cpp:57
>>> #10 0x00007ffff2dcb74d in nsTArrayInfallibleAllocator::Malloc (size=128)
>>>    at ../../dist/include/nsTArray.h:56
>>> ...
>>> 
>>> Printing some variables here:
>>> 
>>> (gdb) up
>>> #1  0x000000000041afdf in arena_bin_malloc_hard (arena=0x7ffff6c00180,
>>> bin=0x7ffff6c007c8)
>>>    at /hack/mozilla-central/memory/jemalloc/src/src/arena.c:1189
>>> 1189            assert(bin->runcur->nfree > 0);
>>> (gdb) p bin
>>> $1 = (arena_bin_t *) 0x7ffff6c007c8
>>> (gdb) p *bin
>>> $2 = {lock = {lock = {__data = {__lock = 1, __count = 0, __owner = 6469,
>>> __nusers = 1,
>>>        __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
>>>      __size = "\001\000\000\000\000\000\000\000E\031\000\000\001", '\000'
>>> <repeats 26 times>, __align = 1}}, runcur = 0x7fffc2ced000, runs = {rbt_root
>>> = 0x7fffca3004d8, rbt_nil = {{
>>>        u = {rb_link = {rbn_left = 0x7ffff6c00800, rbn_right_red =
>>> 0x7ffff6c00800},
>>>          ql_link = {qre_next = 0x7ffff6c00800, qre_prev = 0x7ffff6c00800}},
>>>        prof_ctx = 0x7ffff6c00800}, bits = 0}}, stats = {allocated =
>>> 8896000,
>>>    nmalloc = 197757, ndalloc = 142157, nrequests = 300344, nfills = 5545,
>>>    nflushes = 2848, nruns = 2802, reruns = 5838, curruns = 1143}}
>>> (gdb) p bin->runcur
>>> $3 = (arena_run_t *) 0x7fffc2ced000
>>> (gdb) p *(bin->runcur)
>>> $4 = {bin = 0x7ffff6c007c8, nextind = 4544384, nfree = 0}
>>> 
>>> Any help would be greatly appreciated.
>>> 
>>> Thanks,
>>> Benoit
> _______________________________________________
> jemalloc-discuss mailing list
> jemalloc-discuss at canonware.com
> http://www.canonware.com/mailman/listinfo/jemalloc-discuss
> 