Memory allocation/release hooks

[Shamis, Pavel]

Dear Jemalloc Community,

We are developer of UCX project [1] and as part of the effort
we are looking for a malloc library that supports hooks for alloc/dealloc chunks and can be used for the following:

(a) Allocation of memory that can be shared transparently between processes on the same node. For this purpose we would like to mmap memory with MAP_SHARED. This is very useful for implementation for Remote Memory Access (RMA) operations in MPI-3 one-sided [2] and OpenSHMEM [3] communication libraries. This allow a remote process to map user allocated memory and provide RMA operations through memcpy().

(b) Implementation of memory de-allocation hooks for RDMA hardware (Infiniband, ROCE, iWarp etc.). For optimization purpose we implement a lazy memory de-registration (memory unpinning) policy and we use the hook for the  notification of communication library about memory release event. On the event, we cleanup our registration cache and de-register (unpin) the memory on hardware.

Based on this requirements we would like to understand what is the best approach for integration this functionality within jemalloc.

Regards,
Pasha & Yossi

[1] OpenUCX: https://github.com/openucx/ucx or www.openucx.org
[2] MPI SPEC: http://www.mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf
[3] OpenSHMEM SPEC: http://bongo.cs.uh.edu/site/sites/default/site_files/openshmem-specification-1.2.pdf