calling a function via PLT and jemalloc realloc changes function first argument (XMM0)

[Jason Evans]

On Nov 9, 2013, at 8:30 AM, David Abdurachmanov <david.abdurachmanov at gmail.com> wrote:
> I am having problems with jemalloc 3.4.1 (currently we use 2.2.2 in production). I found that with jemalloc 3.4.1 function first argument will be changed if first argument is passed by XMM0 register. Compiled with GCC 4.8.1 (tested also with 4.8.2). No problems on Scientific Linux 6 (RHEL6-based), but it fails on Scientific Linux 5 (RHEL5-based). All of this is because _dl_lookup_symbol_x calls _realloc_ in Scientific Linux 5.
> 
> This probably makes jemalloc 3.4.1 and the whole 3.X.Y series not recommended for RHEL5 and RHEL5-based distributions.

If I understand correctly, then the simplest correct fix for this problem is to modify ld.so such that it preserves all caller-saved registers when calling out to functions like realloc(3).  In my opinion, ld.so probably shouldn't be using the normal malloc at all (instead use a directly embedded minimal malloc implementation), because there are lots of mind-boggling ways bootstrapping can fail, but that's a more involved change.

Can you disable the gcc optimization (-fno-tree-vectorize) when building jemalloc?  You won't hit actual floating point code in jemalloc unless you enable heap profiling, so that should prevent XMM usage in all the relevant jemalloc code.  If that works okay, I'll need to get a better understanding of when to automatically configure the gcc flags that way when building jemalloc.

What about setting LD_BIND_NOW=1 in the environment so that all the register-corrupting badness happens prior to application execution (when it doesn't matter)?

Thanks,
Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20131113/a164f498/attachment.html>