<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">On Jun 3, 2014, at 11:41 AM, Sheppard Parker <<a href="mailto:Sheppard.Parker@maxpoint.com">Sheppard.Parker@maxpoint.com</a>> wrote:<div><blockquote type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1"><p class="MsoNormal">We have been running a particular jemalloc-aware application on Ubuntu Server 12.04 LTS for about a year now with no trouble (thx btw… we were having major fragmentation issues prior to trying jemalloc). Nominal RES memory footprint is
between 23GB and 27GB depending on how much data is loaded at any given time of the day or day of the week.<o:p></o:p></p><p class="MsoNormal"><o:p> </o:p></p><p class="MsoNormal">We recently purchased some new hardware that came with Ubuntu Server 14.04 LTS and now our same app shows almost 2X the RES memory footprint when loaded with the same data. I don’t think it is growing, so it may not be a big deal, but I
am still concerned. Why the major difference? Exact same executable installed, no code changes, even the same hardware (Dell R620s). Only difference is the move from Ubuntu Server 12.04 (3.8.0-36-generic #52~precise1-Ubuntu SMP) to Ubuntu Server 14.04 (one
machine is 3.13.0-24-generic #47-Ubuntu SMP, another is 3.13.0-27-generic #50-Ubuntu SMP).<o:p></o:p></p><p class="MsoNormal"><o:p> </o:p></p><p class="MsoNormal">Any ideas? Is there something enabled/disabled on Ubuntu 14.04 that I need to modify to get things working like they do on the older 12.04? FWWIW, we have been using jemalloc v3.5.0.<o:p></o:p></p><p class="MsoNormal"><o:p> </o:p></p><p class="MsoNormal">I tried rebuilding our app (again, no code changes) with jemalloc 3.5.1 and 3.6.0. No change. Seems like jemalloc is unhappy about something on Ubuntu 14.04 (or visa versa), but what? Has anybody else encountered anything similar?
<o:p></o:p></p></div></div></blockquote><br></div><div>My only guess is that the newer system is using transparent huge pages, whereas the older one is not. I recently came across this related blog post:</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span><a href="http://dev.nuodb.com/techblog/linux-transparent-huge-pages-jemalloc-and-nuodb">http://dev.nuodb.com/techblog/linux-transparent-huge-pages-jemalloc-and-nuodb</a></div><div><br></div><div>In short, madvise() isn't actually purging unused dirty (4 KiB) pages if the underlying memory has been promoted to huge (2 MiB) pages. In the short term, the solution seems to be disabling transparent huge pages. In the long run I hope the Linux kernel improves its algorithms for such usage patterns, and I'm also contemplating layout strategies that coexist better with huge pages.</div><div><br></div><div>Jason</div></body></html>