hang in jemalloc()

[John Esmet]

Hi Alexey,

I have run into similar issues (exorbitantly long wait times for the malloc
mutex). The only way I could reduce the pain was to either reduce the
number of threads, or reduce the amount of malloc/free() done by the
application.

If you're using a library (like this xml one) it may be hard to reduce the
number of malloc/frees() actually done.

Can you try getting a full set of stack traces on the next stall (for all
threads?). I have a feeling that your particular stack trace may be held up
behind an arena_flush() but it'd be good to know for sure.


On Tue, Jan 21, 2014 at 5:46 AM, Alexey Kuts <akuts at yandex-team.ru> wrote:

> Hi,
>
> sometimes (maybe once per day) we have
> an annoying hang in jemalloc().
>
> jemalloc-version: 3.4.1-0-g0135fb806e4137dc9cdf152541926a2bc95e33f0
> Ubuntu 12.04.3 LTS, 3.2.0-33-generic x86_64 GNU/Linux
>
>
> #0  0x00007f6bd681a89c in __lll_lock_wait () from /lib/x86_64-linux-gnu/
> libpthread.so.0
> #1  0x00007f6bd6816080 in _L_lock_903 () from /lib/x86_64-linux-gnu/
> libpthread.so.0
> #2  0x00007f6bd6815f19 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/
> libpthread.so.0
> #3  0x0000000000459127 in malloc_mutex_lock (mutex=0x7f6bd4400a08) at
> include/jemalloc/internal/mutex.h:77
> #4  tcache_bin_flush_small (tbin=<optimized out>, binind=15, rem=3,
> tcache=0x7f6bd4006000) at src/tcache.c:105
> #5  0x000000000045958d in tcache_event_hard (tcache=0x7f6bd4006000) at
> src/tcache.c:39
> #6  0x0000000000437ebe in tcache_event (tcache=0x7f6bd4006000) at
> include/jemalloc/internal/tcache.h:271
> #7  tcache_alloc_large (size=<optimized out>, tcache=<optimized out>,
> zero=<optimized out>) at include/jemalloc/internal/tcache.h:383
> #8  arena_malloc (zero=false, size=<optimized out>, arena=0x0,
> try_tcache=true) at include/jemalloc/internal/arena.h:944
> #9  imallocx (arena=0x0, try_tcache=true, size=<optimized out>) at
> include/jemalloc/internal/jemalloc_internal.h:798
> #10 imalloc (size=<optimized out>) at include/jemalloc/internal/
> jemalloc_internal.h:807
> #11 jemalloc (size=<optimized out>) at src/jemalloc.c:887
> #12 0x00007f6bd6d3bad8 in xmlStrndup () from /usr/lib/x86_64-linux-gnu/
> libxml2.so.2
> #13 0x00007f6bd6ce3029 in xmlNodeListGetString () from
> /usr/lib/x86_64-linux-gnu/libxml2.so.2
> #14 0x00007f6bd6d0038f in xmlValidateElement () from
> /usr/lib/x86_64-linux-gnu/libxml2.so.2
> #15 0x00007f6bd6d0034e in xmlValidateElement () from
> /usr/lib/x86_64-linux-gnu/libxml2.so.2
> #16 0x00007f6bd6d0034e in xmlValidateElement () from
> /usr/lib/x86_64-linux-gnu/libxml2.so.2
> #17 0x00007f6bd6d00528 in xmlValidateDtd () from /usr/lib/x86_64-linux-gnu/
> libxml2.so.2
> #18 0x00007f6bd7005355 in xmlpp::DtdValidator::validate(xmlpp::Document
> const*) () from /usr/lib/libxml++-2.6.so.2
>
> Will appreciate any hints or suggestions
>
> --
> Kuts Alexey
> _______________________________________________
> jemalloc-discuss mailing list
> jemalloc-discuss at canonware.com
> http://www.canonware.com/mailman/listinfo/jemalloc-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20140122/0936af7d/attachment.html>