RFC: TCMalloc-style new/delete hooks

[Daniel Micay]

On 14/10/14 12:13 PM, David Rigby wrote:
> Hi,
> 
> We are currently using TCMalloc as our memory allocator, however the significantly better fragmentation characteristics and deterministic lowest-available address selection of jemalloc means we want switching to jemalloc in the near future.
> 
> One of (the only?) sticking points however is the lack of a direct equivalent to TCMalloc’s new/delete hooks, which allow an application to register callbacks when memory is allocated/freed by the application. 
> 
> We use this feature to essentially perform sub-heap memory tracking, to determine how much memory different buckets (think tables/databases) are using. To be more specific, as a worker thread is assigned to a particular bucket the bucket ID is stored in TLS, and then when a new/delete callback is invoked we lookup the thread’s current bucket from TLS and increment/decrement the total used as appropriate.
> 
> To allow us to work with jemalloc, I’ve implemented[1] equivalent functionality in jemalloc.
> 
> I did consider making use of the arena functionality in jemalloc for this, but I was concerned about the potential increase in fragment ion with many arenas, which is exactly one of the reasons why we want to move away from TCMalloc (I’m proposing setting narenas=1 when we deploy).
> 
> How would you (Jason?) feel about merging this patch, or something conceptually similar into upstream? 
> 
> 
> Thanks,
> 
> Dave Rigby

It seems like you could accomplish the same thing by setting a jemalloc
prefix and then making wrappers without the prefix externally.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://jemalloc.net/mailman/jemalloc-discuss/attachments/20141014/aea7688c/attachment.sig>