Just wanted to mention that I printed out the addresses being returned from jemalloc() and those addresses fit within the range of the smaps in question. So I'm certain the smaps with large Private_Clean bytes belongs to jemalloc. Questions: 1) Why does jemalloc have smaps with such large Private_Clean size? I'm actually surprised jemalloc has such large smaps in general. I would expect a bunch of smaller smaps that match the configured chunk size. 2) Will the Linux kernel ever reclaim private clean pages, or at least move them out of physical memory? Eventually we see RSS for several of our processes (that use jemalloc) grow beyond what's expected and the OOM killer in the kernel takes out our process(es). 3) Are there any configurations that can be done to influence the handling of private clean pages? Either jemalloc or Linux tuning. thx again ! <table width=100% style="border-collapse:collapse;"> <tr valign=top height=8> <td width=96 style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;">From: <td style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;">Kurtis Martin/Raleigh/IBM <tr valign=top height=8> <td width=96 style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;">To: <td style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;">jemalloc-discuss@canonware.com, <tr valign=top height=8> <td width=96 style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;">Date: <td style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;">05/29/2013 04:52 PM <tr valign=top height=8> <td width=96 style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;">Subject: <td style="border-style:solid;border-color:#000000;border-width:0px 0px 0px 0px;padding:0px 0px;">High amount of private clean data in smaps</table> <hr noshade> We have a java application which has a native component written in C, that makes use of JNI to manage several of our objects that we directly allocate via jemalloc. One of the goals for our C code is to not allow our processes RSS size to grow beyond a certain point. To do this our app monitors jemallocs stats.cactive, when it sees that cactive reach a certain point we write certain some of our objects to disk then we free the allocated memory. In this case, the objects that we are freeing are 4K and above, so when we free the pages holding these objects should no longer be active. That part seems to be working. After several hours of running our application at full load, we start to see our RSS go beyond what is expected. This unexpected growth seems to happen faster for some instances of our application vs others. I compared the /proc smaps file between instances of our app that had a high RSS to those with expected RSS. In both cases smap shows a really large map that I think belongs to jemalloc. I believe it belongs to jemalloc, because if it didn't the sum of jemallocs active size + the RSS size of this map goes well beyond our processes overall RSS size. The thing that is really strange about this mmap is that it's Private_Clean sizes is really high, e.g. 1.4 GBs. I don't know much about private clean pages, other than what a few google searchs told me about them being pages that have not been modified since they were mapped. I'm hopeful that you can tell me more about private clean pages and if/why jemalloc would have maps with such high private clean pages. Here's one such map from my smaps file and the jemalloc stats around the same time. At this time /proc status showed our VmRSS size was 10440804 kB: 7f41dc200000-7f437a400000 rw-p 00000000 00:00 0 Size: 6785024 kB Rss: 4037056 kB Pss: 4037056 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 1499364 kB Private_Dirty: 2537692 kB Referenced: 2536428 kB Swap: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB Jemalloc stats: ___ Begin jemalloc statistics ___ Version: 3.3.1-0-g9ef9d9e8c271cdf14f664b871a8f98c827714784 Assertions disabled Run-time option settings: opt.abort: false opt.lg_chunk: 21 opt.dss: "secondary" opt.narenas: 96 opt.lg_dirty_mult: 6 opt.stats_print: false opt.junk: false opt.quarantine: 0 opt.redzone: false opt.zero: false CPUs: 24 Arenas: 96 Pointer size: 8 Quantum size: 16 Page size: 4096 Min active:dirty page ratio per arena: 64:1 Chunk size: 2097152 (2^21) Allocated: 7350097392, active: 7438028800, mapped: 13549699072 Current active ceiling: 7486832640 chunks: nchunks highchunks curchunks 6927 6622 6461 huge: nmalloc ndalloc allocated 0 0 0 Merged arenas stats: assigned threads: 93 dss allocation precedence: N/A dirty pages: 1815925:2559 active:dirty, 10091062 sweeps, 30421949 madvises, 135567186 purged allocated nmalloc ndalloc nrequests small: 1072125424 106372291 99191380 106372291 large: 6277971968 51052119 50542543 51052119 total: 7350097392 157424410 149733923 157424410 active: 7438028800 mapped: 13545504768 bins: bin size regs pgs allocated nmalloc ndalloc newruns reruns curruns 0 8 501 1 128 30 14 11 0 7 1 16 252 1 672 42 0 11 0 11 2 32 126 1 64 2 0 1 0 1 3 48 84 1 114892800 16709174 14315574 43714 1159683 30488 4 64 63 1 851264 167481 154180 3075 1219 235 5 80 50 1 190426160 36559908 34179581 115227 4236615 50957 6 96 84 2 2112 30 8 11 0 7 [7..10] 11 224 72 4 224 1 0 1 0 1 [12] 13 320 63 5 765952000 52935623 50542023 64485 14712013 40300 [14..27] large: size pages nmalloc ndalloc nrequests curruns [1] 8192 2 30 8 30 22 12288 3 51050157 50542023 51050157 508134 16384 4 1920 512 1920 1408 [13] 73728 18 1 0 1 1 [23] 172032 42 1 0 1 1 [214] 1052672 257 10 0 10 10 [252] --- End jemalloc statistics --- Any help is greatly appreciated .