Segfault with jemalloc 3.4

[Jason Evans]

On Oct 8, 2013, at 10:05 PM, Ofer Samocha <ofers at iMesh.com> wrote:
> I have been using the jemalloc 3.4.0. Usually  it runs fine but i have a rare crash at the following location.
> Any pointer regarding the resolution would be helpful.
>  
> Thanks in advance.
>  
> #0  0x0000000000aa4abe in arena_chunk_purge (arena=0x7ff5398000c0, all=<value optimized out>) at src/arena.c:783
> #1  arena_purge (arena=0x7ff5398000c0, all=<value optimized out>) at src/arena.c:952
> #2  0x0000000000aa5542 in arena_run_trim_tail (ptr=0x7ff52543a000, oldsize=20480, size=<value optimized out>, extra=<value optimized out>, zero=<value optimized out>) at src/arena.c:1165
> #3  arena_ralloc_large_shrink (ptr=0x7ff52543a000, oldsize=20480, size=<value optimized out>, extra=<value optimized out>, zero=<value optimized out>) at src/arena.c:1794
> #4  arena_ralloc_large (ptr=0x7ff52543a000, oldsize=20480, size=<value optimized out>, extra=<value optimized out>, zero=<value optimized out>) at src/arena.c:1909
> #5  arena_ralloc_no_move (ptr=0x7ff52543a000, oldsize=20480, size=<value optimized out>, extra=<value optimized out>, zero=<value optimized out>) at src/arena.c:1951
> #6  0x0000000000aa799d in arena_ralloc (arena=0x0, ptr=0x7ff52543a000, oldsize=32768, size=2985, extra=1024, alignment=16498863232710968265, zero=false, try_tcache_alloc=true, try_tcache_dalloc=true) at src/arena.c:1971
> #7  0x0000000000a9e65d in irallocx (ptr=0x7ff52543a000, size=8462) at include/jemalloc/internal/jemalloc_internal.h:1001
> #8  iralloc (ptr=0x7ff52543a000, size=8462) at include/jemalloc/internal/jemalloc_internal.h:1016
> #9  realloc (ptr=0x7ff52543a000, size=8462) at src/jemalloc.c:1181
> #10 0x00000000008e91ac in reserve (this=0x7ff4d1cdd200) at /home/imesh/SFIM2Rel/src/../ViberCore/include/sti/SmartBuffer.h:138
> #11 resize (this=0x7ff4d1cdd200) at /home/imesh/SFIM2Rel/src/../ViberCore/include/sti/SmartBuffer.h:126
> #12 readFromSocket<sti::IRWPollableObject> (this=0x7ff4d1cdd200) at /home/imesh/SFIM2Rel/src/../ViberCore/include/sti/streaminterface.h:487
> #13 CMsgSockHandlerImpl::recvBuffer (this=0x7ff4d1cdd200) at /home/imesh/SFIM2Rel/src/comm/MsgSockHandler.cpp:566
> #14 0x00000000008ed77c in CMsgSockHandlerImpl::HandleInput (this=0x7ff4d1cdd200) at /home/imesh/SFIM2Rel/src/comm/MsgSockHandler.cpp:290
> #15 0x0000000000870d94 in sti::CSocketAttachedEventHandler<sti::IRWPollableObject>::HandleInput (this=0x7ff502a62480, sock=<value optimized out>) at /home/imesh/SFIM2Rel/src/../ViberCore/include/sti/SocketEventHandler.h:362
> #16 0x00000000008734ce in sti::CSocketEventHandler<sti::IRWPollableObject>::disHandleInput (this=0x7ff502a62480, sock=<value optimized out>) at /home/imesh/SFIM2Rel/src/../ViberCore/include/sti/SocketEventHandler.h:175
> #17 0x00000000009d655f in sti::CDispatcher::HandleEvents (this=0x7ff53942b300, timeout=0) at /home/imesh/SFIM2Rel/src/infra/Dispatcher.cpp:551
> #18 0x000000000080db3b in CServerHelper::HandleEvents (this=0x7ff539421020) at /home/imesh/SFIM2Rel/src/servercomm/ServerHelper.cpp:397
> #19 0x00000000005c8e6e in main (argc=<value optimized out>, argv=<value optimized out>) at /home/imesh/SFIM2Rel/src/SN/SN.cpp:34

It looks like the only way jemalloc can crash here is if the chunk pointer is pointing to unmapped memory, in which case some really bad memory corruption has occurred.  I don't have any good working theories as to how this can happen.  Just how similar do the backtraces look when you see this crash (e.g. is shrinking realloc() always involved)?  What else can you tell me about the app re: number of threads, memory usage, etc.?

Thanks,
Jason

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20131013/8b1e1b49/attachment.html>