Memory usage regression

Mike Hommey mh+jemalloc at glandium.org
Tue Oct 30 08:35:02 PDT 2012


On Fri, Oct 26, 2012 at 06:10:13PM +0200, Mike Hommey wrote:
> > > > For reference, the unzoomed graph looks like this:
> > > > http://i.imgur.com/PViYm.png
> > > 
> > > I rediscovered --enable-munmap, and tried again with that, thinking it
> > > could be related, and it did change something, but it's still growing:
> > > http://i.imgur.com/lWzhG.png
> > 
> > Needless to say, the increases I was observing closely on the the zoomed
> > graph without a matching decrease was entirely due to munmap. Now I need
> > to find the remainder...
> 
> I tested size class independently, and none would cause the VM leak
> alone. Combining small and large classes do, but large + huge or small +
> huge don't.

Some more data: all non-unmapped chunks *are* used to some extent. The
following is a dump of the number of requested and usable bytes in each
chunk ; that's 18M spread across 600M... that sounds like a really bad
case of fragmentation.

Total          17775010 19274192
0x7fffc9d00000 9256 12288
0x7fffca400000 227199 230304
0x7fffca500000 74269 77904
0x7fffca600000 37888 40960
0x7fffca700000 69336 69504
0x7fffca900000 179834 180960
0x7fffcaa00000 2950 3760
0x7fffcab00000 31939 32400
0x7fffcac00000 35215 35392
0x7fffcad00000 12286 12512
0x7fffcae00000 43560 49664
0x7fffcaf00000 32036 32512
0x7fffcb000000 15738 17936
0x7fffcb100000 112 128
0x7fffcb200000 540 576
0x7fffcb300000 514 576
0x7fffcb600000 200 224
0x7fffcb700000 5120 8192
0x7fffcb900000 416 464
0x7fffcba00000 20560 32768
0x7fffcbc00000 4328 4448
0x7fffcbd00000 956 992
0x7fffcbe00000 4992 4992
0x7fffcbf00000 6948 10768
0x7fffcc000000 10082 16800
0x7fffcc100000 160 160
0x7fffcc400000 4216 4224
0x7fffcc500000 5400 8448
0x7fffcc600000 19008 24704
0x7fffcc700000 4964 8592
0x7fffcc800000 98412 98976
0x7fffcc900000 5272 8320
0x7fffcca00000 25612 28560
0x7fffccd00000 25560 25968
0x7fffcce00000 18624 19360
0x7fffccf00000 5908 9024
0x7fffcd000000 202124 205168
0x7fffcd100000 74278 89680
0x7fffcd200000 44 64
0x7fffcd300000 18400 20736
0x7fffcd400000 51612 55120
0x7fffcd500000 3120 3328
0x7fffcd800000 1777 1824
0x7fffcd900000 245152 255072
0x7fffcde00000 16507 21712
0x7fffce000000 1480 1600
0x7fffce200000 12488 19984
0x7fffce300000 8548 11840
0x7fffce400000 5516 9264
0x7fffce500000 25929 35616
0x7fffce600000 13549 20416
0x7fffce700000 8991 11328
0x7fffce800000 35848 38896
0x7fffce900000 27638 32192
0x7fffcea00000 15619 24736
0x7fffceb00000 5438 8512
0x7fffcec00000 7808 8000
0x7fffced00000 12376 19664
0x7fffcee00000 25415 29888
0x7fffcef00000 15664 25440
0x7fffcf000000 4584 8192
0x7fffcf100000 7384 9440
0x7fffcf200000 11032 17136
0x7fffcf300000 11660 14256
0x7fffcf400000 7442 10592
0x7fffcf500000 2568 2704
0x7fffcf600000 25588 31200
0x7fffcf700000 6012 9088
0x7fffcf800000 39 48
0x7fffcf900000 28456 34560
0x7fffcfa00000 10723 16848
0x7fffcfb00000 12491 18608
0x7fffcfc00000 6580 7072
0x7fffcfd00000 6384 6496
0x7fffcff00000 3779 4160
0x7fffd0000000 170 192
0x7fffd0100000 17996 21168
0x7fffd0200000 5328 8368
0x7fffd0300000 14226 20416
0x7fffd0600000 7472 10592
0x7fffd0700000 5280 8320
0x7fffd0800000 39149 48440
0x7fffd0900000 38540 45376
0x7fffd0a00000 44906 51152
0x7fffd0c00000 2612 2736
0x7fffd0d00000 16468 25632
0x7fffd0e00000 2498 2560
0x7fffd0f00000 9442 12704
0x7fffd1000000 13824 19328
0x7fffd1100000 10984 17104
0x7fffd1200000 10368 16432
0x7fffd1300000 928 1072
0x7fffd1500000 432 480
0x7fffd1600000 2072 2176
0x7fffd1700000 6360 9536
0x7fffd1800000 1065 1216
0x7fffd1900000 7816 10976
0x7fffd1a00000 5388 8448
0x7fffd1b00000 1184 1264
0x7fffd1c00000 1990 2208
0x7fffd1d00000 11944 18048
0x7fffd1e00000 5236 8272
0x7fffd1f00000 286 304
0x7fffd2100000 5486 8544
0x7fffd2200000 529 576
0x7fffd2400000 164 192
0x7fffd2500000 4468 4656
0x7fffd2700000 236 256
0x7fffd2800000 784 864
0x7fffd2900000 13164 19472
0x7fffd2a00000 7142 10432
0x7fffd2c00000 8195 11472
0x7fffd2d00000 2722 2848
0x7fffd2f00000 108 112
0x7fffd3000000 5728 8768
0x7fffd3100000 336 336
0x7fffd3200000 163 176
0x7fffd3300000 651 704
0x7fffd3400000 531046 534256
0x7fffd3500000 2074 2144
0x7fffd3700000 1952 2160
0x7fffd3900000 10896 16960
0x7fffd3b00000 10904 12984
0x7fffd3d00000 822 920
0x7fffd3f00000 5256 8288
0x7fffd4000000 1980 2128
0x7fffd4100000 904 960
0x7fffd4200000 2334 2352
0x7fffd4300000 6772 9872
0x7fffd4400000 736 784
0x7fffd4500000 9542 13056
0x7fffd4600000 5552 5568
0x7fffd4700000 2792 2944
0x7fffd4900000 20216 24320
0x7fffd4a00000 3448 3584
0x7fffd4b00000 5296 8336
0x7fffd4c00000 58000 58464
0x7fffd4e00000 5417 5968
0x7fffd4f00000 4124 4848
0x7fffd5000000 5951 8400
0x7fffd5100000 1024 1104
0x7fffd5200000 562 592
0x7fffd5300000 2529 2816
0x7fffd5400000 2090 2112
0x7fffd5500000 369 448
0x7fffd5600000 5160 8192
0x7fffd5800000 1314 1344
0x7fffd5900000 1762 1920
0x7fffd5b00000 2466 2624
0x7fffd5c00000 15512 17216
0x7fffd5d00000 462 464
0x7fffd5e00000 144 144
0x7fffd5f00000 162 176
0x7fffd6000000 264 272
0x7fffd6300000 1404 1472
0x7fffd6400000 1760 1920
0x7fffd6500000 66663 68128
0x7fffd6600000 271 320
0x7fffd6800000 191 224
0x7fffd6900000 96 96
0x7fffd6c00000 992 1088
0x7fffd6d00000 1413 1520
0x7fffd6e00000 720 720
0x7fffd7000000 1102 1120
0x7fffd7100000 1672 1760
0x7fffd7200000 416 432
0x7fffd7300000 3706 4304
0x7fffd7400000 682 720
0x7fffd7500000 328 352
0x7fffd7600000 2372 2544
0x7fffd7700000 3588 3824
0x7fffd7800000 328 336
0x7fffd7900000 859 880
0x7fffd7a00000 7750 10880
0x7fffd7e00000 4009 4192
0x7fffd7f00000 384 384
0x7fffd8000000 2064 2208
0x7fffd8100000 856 864
0x7fffd8200000 1024 1072
0x7fffd8300000 224 224
0x7fffd8400000 614 704
0x7fffd8600000 640 672
0x7fffd8700000 938 1024
0x7fffd8800000 2328 2400
0x7fffd8900000 5160 8192
0x7fffd8a00000 546 576
0x7fffd8b00000 2360 2656
0x7fffd8d00000 7503 10624
0x7fffd8e00000 1146 1216
0x7fffd8f00000 13126 13184
0x7fffd9000000 96 96
0x7fffd9100000 10830 16960
0x7fffd9200000 1744 1920
0x7fffd9300000 3230 3648
0x7fffd9400000 11346 14640
0x7fffd9500000 12128 18464
0x7fffd9600000 2649 2768
0x7fffd9700000 53748 56096
0x7fffd9800000 584 624
0x7fffd9900000 1048 1072
0x7fffd9a00000 1104 1104
0x7fffd9b00000 232 240
0x7fffd9d00000 336 352
0x7fffd9e00000 808 832
0x7fffd9f00000 60208 66288
0x7fffda000000 432 432
0x7fffda100000 1762 1856
0x7fffda200000 62 64
0x7fffda300000 516 576
0x7fffda400000 282 320
0x7fffda500000 6696 9792
0x7fffda600000 828 848
0x7fffda700000 144 176
0x7fffda800000 280 288
0x7fffda900000 512 512
0x7fffdab00000 576 576
0x7fffdac00000 304 304
0x7fffdad00000 192 192
0x7fffdae00000 184 192
0x7fffdaf00000 132 176
0x7fffdb000000 88 112
0x7fffdb100000 5350 8400
0x7fffdb200000 1640 1696
0x7fffdb400000 3022 3200
0x7fffdb500000 796 816
0x7fffdb600000 24 32
0x7fffdb800000 15024 18096
0x7fffdba00000 1600 1648
0x7fffdbb00000 192 192
0x7fffdbc00000 51164 53536
0x7fffdbd00000 550 704
0x7fffdbe00000 24 32
0x7fffdbf00000 33860 34048
0x7fffdc000000 7360 7952
0x7fffdc200000 2068 2120
0x7fffdc300000 6310 6320
0x7fffdc400000 1424 1504
0x7fffdc500000 848 912
0x7fffdc600000 2112 2160
0x7fffdc800000 49248 49248
0x7fffdca00000 704 744
0x7fffdcc00000 8400 12480
0x7fffdce00000 2540 2584
0x7fffdcf00000 5178 5408
0x7fffdd000000 1006 1072
0x7fffdd100000 792 848
0x7fffdd300000 828 928
0x7fffdd400000 9612 10080
0x7fffdd500000 2096 2256
0x7fffdd600000 15889 16864
0x7fffdd700000 2928 3216
0x7fffdd800000 948 976
0x7fffdd900000 5968 6064
0x7fffdda00000 17128 17168
0x7fffddb00000 576 600
0x7fffddc00000 1578 1680
0x7fffddd00000 744 768
0x7fffdde00000 532 544
0x7fffddf00000 8980 13072
0x7fffde000000 5496 8528
0x7fffde100000 824 896
0x7fffde200000 3072 3232
0x7fffde300000 2004 2176
0x7fffde400000 768 768
0x7fffde500000 10736 14976
0x7fffde600000 344 384
0x7fffde700000 960 976
0x7fffde800000 784 848
0x7fffde900000 1280 1296
0x7fffdeb00000 48 48
0x7fffdec00000 5028 5128
0x7fffded00000 1040 1072
0x7fffdee00000 1290 1392
0x7fffdef00000 1024 1024
0x7fffdf100000 1320 1344
0x7fffdf200000 256 272
0x7fffdf500000 1935 2240
0x7fffdf600000 24 32
0x7fffdf700000 1814 1872
0x7fffdf800000 208 240
0x7fffdf900000 476 496
0x7fffdfa00000 2312 2336
0x7fffdfb00000 3752 3888
0x7fffdfc00000 16024 16032
0x7fffdfd00000 6568 6576
0x7fffdfe00000 848 864
0x7fffdff00000 48 48
0x7fffe0000000 2671 3072
0x7fffe0100000 288 304
0x7fffe0200000 2412 2624
0x7fffe0300000 2344 2464
0x7fffe0400000 1904 2048
0x7fffe0500000 1540 1632
0x7fffe0600000 2696 2960
0x7fffe0700000 4828 5312
0x7fffe0900000 12151 16000
0x7fffe0a00000 793 864
0x7fffe0b00000 1738 1952
0x7fffe0c00000 4404 4704
0x7fffe0d00000 48 48
0x7fffe0e00000 11018 14112
0x7fffe0f00000 600 640
0x7fffe1000000 9936 12992
0x7fffe1100000 3820 4000
0x7fffe1200000 136 144
0x7fffe1400000 1871 2016
0x7fffe1500000 192 192
0x7fffe1600000 1686 1792
0x7fffe1700000 624 624
0x7fffe1800000 1072 1152
0x7fffe1900000 288 288
0x7fffe1a00000 1162 1200
0x7fffe1b00000 2096 2096
0x7fffe1c00000 46 64
0x7fffe1d00000 2384 2384
0x7fffe1f00000 1472 1520
0x7fffe2000000 10191 13440
0x7fffe2200000 2072 2080
0x7fffe2300000 48 48
0x7fffe2500000 1357 1408
0x7fffe2600000 8216 12288
0x7fffe2700000 5317 5792
0x7fffe2800000 262 288
0x7fffe2900000 1591 1664
0x7fffe2b00000 1020 1064
0x7fffe2c00000 1430 1600
0x7fffe2d00000 2806 3072
0x7fffe2e00000 2073 2312
0x7fffe2f00000 574 608
0x7fffe3000000 11196 12096
0x7fffe3100000 1176 1216
0x7fffe3200000 192 192
0x7fffe3300000 4167 4256
0x7fffe3400000 301 320
0x7fffe3500000 4272 4304
0x7fffe3600000 2544 2592
0x7fffe3700000 3468 3952
0x7fffe3900000 2135 2320
0x7fffe3a00000 6662 6928
0x7fffe3b00000 1167 1328
0x7fffe3c00000 3771 3856
0x7fffe3d00000 9554 13680
0x7fffe3f00000 1337 1408
0x7fffe4000000 76 80
0x7fffe4100000 1332 1376
0x7fffe4200000 450 496
0x7fffe4300000 792 816
0x7fffe4400000 2794 2848
0x7fffe4500000 944 944
0x7fffe4600000 1189 1232
0x7fffe4700000 740 816
0x7fffe4800000 8888 12984
0x7fffe4900000 4277 4784
0x7fffe4a00000 32 48
0x7fffe4c00000 117 144
0x7fffe4d00000 8856 12944
0x7fffe4f00000 4629 4808
0x7fffe5000000 840 928
0x7fffe5200000 2840 3088
0x7fffe5300000 5720 5968
0x7fffe5400000 7822 7984
0x7fffe5500000 2116 2256
0x7fffe5600000 9130 13296
0x7fffe5900000 140 176
0x7fffe5a00000 4576 4576
0x7fffe5b00000 3166 3216
0x7fffe5c00000 1852 2000
0x7fffe5d00000 318 336
0x7fffe5e00000 3452 3680
0x7fffe5f00000 3637 3744
0x7fffe6000000 9960 11088
0x7fffe6100000 3173 3280
0x7fffe6200000 12221 12496
0x7fffe6400000 912 976
0x7fffe6500000 8752 11920
0x7fffe6600000 448 448
0x7fffe6700000 964 992
0x7fffe6800000 2092 2112
0x7fffe6900000 202 224
0x7fffe6a00000 140 192
0x7fffe6b00000 467 480
0x7fffe6c00000 2088 2096
0x7fffe6d00000 20638 23488
0x7fffe6e00000 9634 13952
0x7fffe6f00000 10182 14368
0x7fffe7000000 168 208
0x7fffe7100000 1578 1656
0x7fffe7200000 3434 3488
0x7fffe7500000 10060 14144
0x7fffe7600000 86 112
0x7fffe7700000 16744 22512
0x7fffe7800000 10748 15056
0x7fffe7900000 6198 6496
0x7fffe7a00000 179 200
0x7fffe7b00000 1763 1792
0x7fffe7c00000 9267 9504
0x7fffe7d00000 1356 1456
0x7fffe7e00000 2627 2728
0x7fffe7f00000 3884 4000
0x7fffe8000000 9835 10448
0x7fffe8100000 936 976
0x7fffe8200000 24208 24224
0x7fffe8300000 15289 15552
0x7fffe8400000 4815 4960
0x7fffe8600000 6542 8648
0x7fffe8700000 2029 2144
0x7fffe8800000 2594 2688
0x7fffe8900000 2696 2752
0x7fffe8a00000 4570 4736
0x7fffe8b00000 2873 3048
0x7fffe8c00000 824 864
0x7fffe8d00000 11827 16960
0x7fffe8e00000 52611 56560
0x7fffe8f00000 2066 2080
0x7fffe9000000 295 304
0x7fffe9100000 1241 1384
0x7fffe9200000 1606 1792
0x7fffe9300000 15386 20144
0x7fffe9400000 992 1024
0x7fffe9500000 10745 15040
0x7fffe9700000 4194 4456
0x7fffe9800000 16174 16240
0x7fffe9900000 28125 32240
0x7fffe9a00000 4499 5040
0x7fffe9b00000 917 1024
0x7fffe9d00000 8886 9344
0x7fffe9e00000 6977 7512
0x7fffe9f00000 6957 7288
0x7fffea000000 3278 3360
0x7fffea100000 384 416
0x7fffea300000 1122 1264
0x7fffea400000 9431 13568
0x7fffea500000 1271 1392
0x7fffea700000 4045 4176
0x7fffea800000 4231 5048
0x7fffeaa00000 1671 1920
0x7fffeab00000 3209 3248
0x7fffeac00000 1893 2120
0x7fffead00000 140 176
0x7fffeae00000 8388 12480
0x7fffeaf00000 9073 13184
0x7fffeb000000 3944 4112
0x7fffeb100000 720 768
0x7fffeb200000 23896 28176
0x7fffeb300000 1176 1216
0x7fffeb400000 8728 12816
0x7fffeb500000 10094 10160
0x7fffeb600000 321 352
0x7fffeb700000 1305 1344
0x7fffeb800000 4544 4544
0x7fffeb900000 14949 19112
0x7fffeba00000 9484 13584
0x7fffebb00000 76407 80584
0x7fffebc00000 1088 1104
0x7fffebd00000 23516 33792
0x7fffebf00000 12708 16864
0x7fffec000000 12222 12272
0x7fffec100000 9335 9504
0x7fffec200000 4193 4360
0x7fffec300000 2664 3008
0x7fffec400000 837 944
0x7fffec500000 987 1056
0x7fffec700000 570 616
0x7fffec800000 6289 7040
0x7fffec900000 2528 2624
0x7fffeca00000 9089 13296
0x7fffecb00000 1456 1520
0x7fffecd00000 5710 5792
0x7fffece00000 2035 2232
0x7fffecf00000 240 272
0x7fffed000000 8418 12512
0x7fffed100000 662 720
0x7fffed200000 1938 2304
0x7fffed300000 7401 10776
0x7fffed400000 272 384
0x7fffed500000 1178 1240
0x7fffed600000 410 512
0x7fffed700000 3556 3600
0x7fffed900000 11648 15744
0x7fffeda00000 8352 12432
0x7fffedb00000 8480 12560
0x7fffedc00000 78 80
0x7fffedd00000 1494 1536
0x7fffede00000 147 160
0x7fffedf00000 1586 1720
0x7fffee000000 2367 2416
0x7fffee200000 626 640
0x7fffee300000 820 896
0x7fffee400000 4854 4992
0x7fffee500000 4459 4816
0x7fffee600000 4208 4600
0x7fffee800000 9865 14016
0x7fffee900000 8289 8504
0x7fffeea00000 2292 2432
0x7fffeec00000 108934 113312
0x7fffeed00000 3624 3712
0x7fffeef00000 200 256
0x7fffef000000 1536 1584
0x7fffef100000 3402 3504
0x7fffef200000 3633 3728
0x7fffef300000 8789 12880
0x7fffef400000 2027 2080
0x7fffef500000 1096 1200
0x7fffef600000 2800 2928
0x7fffef800000 1562 1584
0x7fffef900000 9212 13360
0x7fffefa00000 547 600
0x7fffefc00000 15466 19712
0x7fffefd00000 1969 2064
0x7fffeff00000 3625 3672
0x7ffff0000000 3617 3856
0x7ffff0100000 4391 4448
0x7ffff0200000 7779 7984
0x7ffff0300000 679 768
0x7ffff0400000 20910 25488
0x7ffff0500000 400 416
0x7ffff0700000 103 128
0x7ffff0800000 3442 3536
0x7ffff0900000 13294 13992
0x7ffff0a00000 2699 2848
0x7ffff0b00000 4930 5120
0x7ffff0d00000 3415 3584
0x7ffff0e00000 1144 1368
0x7ffff0f00000 5999 6136
0x7ffff1000000 262 320
0x7ffff1200000 475 520
0x7ffff1300000 10972 15168
0x7ffff1500000 9207 9352
0x7ffff1600000 3177 3232
0x7ffff1700000 1943 2032
0x7ffff1800000 9582 13792
0x7ffff1900000 13063 13792
0x7ffff1a00000 4125 4296
0x7ffff1b00000 3745 4000
0x7ffff1c00000 8916 9040
0x7ffff1d00000 11332 12008
0x7ffff1f00000 4474 4880
0x7ffff2000000 28100 28856
0x7ffff2100000 28128 28712
0x7ffff2200000 4719 5072
0x7ffff2300000 12907 13312
0x7ffff2400000 199185 204264
0x7ffff2500000 78610 79008
0x7ffff2600000 19306 21200
0x7ffff2800000 27222 29088
0x7ffff2a00000 57133 61184
0x7ffff2b00000 80276 86944
0x7ffff2c00000 77393 81712
0x7ffff2d00000 13990 14576
0x7ffff2e00000 66713 68416
0x7ffff2f00000 30021 31232
0x7ffff3000000 26749 28472
0x7ffff3200000 44083 48880
0x7ffff3300000 77406 88464
0x7ffff3400000 42028 45504
0x7ffff3500000 123067 130704
0x7ffff3600000 388411 433344
0x7ffff3700000 395081 430416
0x7ffff3900000 380507 414240
0x7ffff3a00000 527976 567536
0x7ffff3b00000 708191 768544
0x7ffff3c00000 205120 208896
0x7ffff3d00000 148049 160072
0x7ffff3e00000 24953 25616
0x7ffff3f00000 106595 119072
0x7ffff4100000 337114 354976
0x7ffff4200000 440608 475552
0x7ffff4300000 431193 460304
0x7ffff4400000 186405 193344
0x7ffff4500000 338115 371912
0x7ffff4600000 360147 384888
0x7ffff4800000 431303 470800
0x7ffff4900000 499848 543760
0x7ffff4b00000 163631 176672
0x7ffff4c00000 59282 60464
0x7ffff4d00000 50310 57584
0x7ffff4e00000 105842 116624
0x7ffff4f00000 16718 17808
0x7ffff5000000 41049 42576
0x7ffff5100000 20971 22592
0x7ffff5200000 25411 31344
0x7ffff5400000 204285 220016
0x7ffff5500000 166705 178128
0x7ffff5700000 121964 135328
0x7ffff5800000 30275 32016
0x7ffff5900000 37462 40336
0x7ffff5a00000 105017 116672
0x7ffff5b00000 270094 290112
0x7ffff5c00000 69549 79648
0x7ffff5d00000 77453 84128
0x7ffff5e00000 15851 17288
0x7ffff5f00000 108060 117696
0x7ffff6000000 539778 549008
0x7ffff6200000 540620 580592
0x7ffff6300000 893387 902352
0x7ffff6400000 1000202 1006432
0x7ffff6500000 397745 432464
0x7ffff6e00000 670342 714544

Mike



More information about the jemalloc-discuss mailing list