analysing jemalloc's stats (jemalloc tuning help redux)

[David Rigby]

Hi,

Insipired by Jason’s post [1] explaining how to interpret jemalloc’s size class stats and the resulting table, I threw together a quick script to essentially output the same information from an arbitrary malloc_stats_print output. 

I figured this might be useful to others, so it’s at https://github.com/daverigby/scripts/blob/master/jemalloc_analyse.py 

Example output (*very* similar to Jason’s):

—cut—
=== Stats for Arena 'merged' ===
small allocation stats:
  bin   size (B)  regions      pages    allocated (B)  cur runs       % of small               % of blame
                  per run      per run                         utilization    frag memory (B)                         

  0     8         501          1        5032           3       42%    0%      6992             0%      
  1     16        252          1        24480          8       76%    0%      7776             0%      
  2     32        126          1        1428352        360     98%    9%      23168            1%      
  3     48        84           1        73152          37      49%    0%      76032            2%      
  4     64        63           1        892096         235     94%    6%      55424            2%      
  5     80        50           1        198560         54      92%    1%      17440            1%      
  6     96        84           2        274464         35      97%    2%      7776             0%      
  7     112       72           2        40320          7       71%    0%      16128            0%      
  8     128       63           2        157568         24      81%    1%      35968            1%      
  9     160       51           2        633760         79      98%    4%      10880            0%      
  10    192       63           3        136704         15      75%    1%      44736            1%      
  11    224       72           4        80416          9       55%    1%      64736            2%      
  12    256       63           4        153344         12      79%    1%      40192            1%      
  13    320       63           5        878720         47      93%    6%      68800            2%      
  14    384       63           6        133248         9       61%    1%      84480            2%      
  15    448       63           7        1523648        57      95%    10%     85120            2%      
  16    512       63           8        963072         38      79%    6%      262656           8%      
  17    640       51           8        188800         9       64%    1%      104960           3%      
  18    768       47           9        420864         13      90%    3%      48384            1%      
  19    896       45           10       1169280        32      91%    7%      120960           3%      
  20    1024      63           16       355328         9       61%    2%      225280           6%      
  21    1280      51           16       1602560        27      91%    10%     160000           5%      
  22    1536      42           16       456192         8       88%    3%      59904            2%      
  23    1792      38           17       225792         7       47%    1%      250880           7%      
  24    2048      65           33       1103872        11      75%    7%      360448           10%     
  25    2560      52           33       737280         10      55%    5%      593920           17%     
  26    3072      43           33       866304         8       82%    5%      190464           5%      
  27    3584      39           35       1075200        11      70%    7%      462336           13%     

  total                                 15.1 MB                               3.3 MB                   

    utilization = allocated / (size * regions_per_run * cur runs)
    % of small  = allocated / total allocated
    frag memory = (size * regions_per_run * cur runs) - allocated
    % of blame  = frag memory / total frag memory
—cut—

Thanks,

Dave


[1]: http://jemalloc.net/mailman/jemalloc-discuss/2013-November/000675.html