Excessive VM usage with jemalloc

[Abhishek Singh]

Hi

We are trying to replace glibc malloc with jemalloc because we have several
concurrent allocations and in all our benchmarks jemalloc is consistently
better than glibc malloc and many others.

Our setups start typically with 96 GB of RAM and up. We have observed that
using jemalloc the virtual memory usage of our process rises up to around
75GB. While the resident memory stays low and it is not a problem as such,
when we try to fork a process from within here, it fails as the kernel
assumes there is not enough memory to copy the VM space. Perhaps a vfork
would be better but we can't use that for now.

So we have made some modifications to jemalloc so that all huge memory
allocations are forced to unmap their memory on freeing up. Non huge memory
allocations and freeing up remain the same. This seems to help us. I have
attached the patch here which is against jemalloc-3.3.1. Please review and
suggest if there is a better way to handle this.

-- 
Regards
Abhishek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20130429/ad69bdc4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jemalloc.patch
Type: application/octet-stream
Size: 5701 bytes
Desc: not available
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20130429/ad69bdc4/attachment.obj>