<html><head></head><body><div class="ydpcd03535byahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div></div>
<div dir="ltr" data-setdir="false"><div><div dir="ltr" data-setdir="false" style="color: rgb(0, 0, 0); font-family: Helvetica Neue, Helvetica, Arial, sans-serif;">So you need to refernce the old docs, <a href="http://erlang.org/documentation/doc-9.3/lib/tools-2.11.2/doc/html/instrument.html" style="color: rgb(25, 106, 212); text-decoration-line: underline;" rel="nofollow" target="_blank">http://erlang.org/documentation/doc-9.3/lib/tools-2.11.2/doc/html/instrument.html</a>.<br><br></div><div dir="ltr" data-setdir="false" style="color: rgb(0, 0, 0); font-family: Helvetica Neue, Helvetica, Arial, sans-serif;">memory_data() will return a list of all the allocations, find the binary heap in there and itl show which memory address the terms are stored at. You can then check the processes memory /proc/#{:os.pid()}/mem at that address for the actual term.<br><br>Also a few useful hipe bifs as well that got removed also <span><a href="https://github.com/erlang/otp/blob/master/erts/emulator/hipe/hipe_bif2.tab" rel="nofollow" target="_blank" class="">https://github.com/erlang/otp/blob/master/erts/emulator/hipe/hipe_bif2.tab</a><br><br></span><div><br></div><div><br></div><div><br></div></div></div></div><div><br></div>
</div><div id="yahoo_quoted_6753781743" class="yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
On Thursday, April 9, 2020, 08:43:49 a.m. EDT, Frank Muller <frank.muller.erl@gmail.com> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div id="yiv2278926173"><div><div><div>Vans,</div><div><br clear="none"></div><div>Can you please share how to track these large binaries with let’s say Erlang 20 and/or HIPE?</div><div><br clear="none"></div><div>This will be useful to all of us and Devon can certainly downgrade to Erlang 20 to at least identify the culprit. </div><div><br clear="none"></div><div>/Frank</div></div><div class="yiv2278926173yqt9990132159" id="yiv2278926173yqt97723"><div><br clear="none"><div class="yiv2278926173gmail_quote"><div class="yiv2278926173gmail_attr" dir="ltr"><br clear="none"></div><blockquote class="yiv2278926173gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204);"><div><div style="font-family:Helvetica, Arial, sans-serif;font-size:13px;"><div style="font-family:Helvetica, Arial, sans-serif;"></div>
<div dir="ltr" style="font-family:Helvetica, Arial, sans-serif;">This used to be possible via the instrument module but the functionality got removed in OTP21, I used this functionality to detect a large binheap leak in jiffy way back when. Theres a lengthy discussion here under a PR called "improve instrumentation" but it really just "removed instrumentation" <span style="font-family:Helvetica, Arial, sans-serif;"><a rel="nofollow" shape="rect" target="_blank" href="https://github.com/erlang/otp/pull/1790" style="font-family:Helvetica, Arial, sans-serif;">https://github.com/erlang/otp/pull/1790</a>.<br clear="none"></span><div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><div dir="ltr" style="font-family:Helvetica, Arial, sans-serif;">There are BIF debug functions under hipe that can print this information. hipe_bifs:show_heap/1 and friends.</div></div>
</div></div><div><div id="yiv2278926173m_-4622445273957406379yahoo_quoted_6777507682">
<div style="font-family:Helvetica, Arial, sans-serif;font-size:13px;color:rgb(38,40,42);">
<div style="font-family:Helvetica, Arial, sans-serif;">
On Wednesday, April 8, 2020, 02:53:49 p.m. EDT, Dániel Szoboszlay <<a rel="nofollow" shape="rect" ymailto="mailto:dszoboszlay@gmail.com" target="_blank" href="mailto:dszoboszlay@gmail.com" style="font-family:Helvetica, Arial, sans-serif;">dszoboszlay@gmail.com</a>> wrote:
</div>
<div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div>
<div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div>
<div style="font-family:Helvetica, Arial, sans-serif;"><div id="yiv2278926173m_-4622445273957406379yiv5226392344" style="font-family:Helvetica, Arial, sans-serif;"><div style="font-family:Helvetica, Arial, sans-serif;"><div dir="ltr" style="font-family:Helvetica, Arial, sans-serif;">Hi,<div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><div style="font-family:Helvetica, Arial, sans-serif;">Even though you cannot trace on allocating large binaries, you may try tracing garbage collections, and look for GC-s that clean up a lot of off-heap binary data. This could at least narrow done the search for some processes, although it won't tell you where the allocation happens. But maybe once you know which processes are guilty you will be able to add more targeted tracing until you find the root cause.</div><div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><div style="font-family:Helvetica, Arial, sans-serif;">Cheers,</div><div style="font-family:Helvetica, Arial, sans-serif;">Daniel</div></div><br clear="none"><div id="yiv2278926173m_-4622445273957406379yiv5226392344yqt83720" style="font-family:Helvetica, Arial, sans-serif;"><div style="font-family:Helvetica, Arial, sans-serif;"><div dir="ltr" style="font-family:Helvetica, Arial, sans-serif;">On Wed, 8 Apr 2020 at 16:14, Lukas Larsson <<a rel="nofollow" shape="rect" ymailto="mailto:lukas@erlang.org" target="_blank" href="mailto:lukas@erlang.org" style="font-family:Helvetica, Arial, sans-serif;">lukas@erlang.org</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;font-family:Helvetica, Arial, sans-serif;border-left-color:rgb(204,204,204);"><div dir="ltr" style="font-family:Helvetica, Arial, sans-serif;"><div style="font-family:Helvetica, Arial, sans-serif;">Hello,</div><br clear="none"><div style="font-family:Helvetica, Arial, sans-serif;"><div dir="ltr" style="font-family:Helvetica, Arial, sans-serif;">On Tue, Apr 7, 2020 at 9:26 AM Devon Estes <<a rel="nofollow" shape="rect" ymailto="mailto:devon.c.estes@gmail.com" target="_blank" href="mailto:devon.c.estes@gmail.com" style="font-family:Helvetica, Arial, sans-serif;">devon.c.estes@gmail.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;font-family:Helvetica, Arial, sans-serif;border-left-color:rgb(204,204,204);"><div style="font-family:Helvetica, Arial, sans-serif;">Hi all,</div><div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><div style="font-family:Helvetica, Arial, sans-serif;">I’m seeing some cases in my application where our off-process binary heap allocation jumps by several orders of magnitude and then goes down right after. I’m sure this is something that’s in our app just loading dozens of huge binaries into memory at once and not a bug in anything underlying or a binary leak, but finding where these allocations are happening so I can make some changes to avoid this has so far not yielded any results. Ideally I’d like to be able to set a trace with something like erlang:trace/3 on some function that sends a tracer message whenever a binary over 30MB is allocated and includes the call stack or even just the calling function that allocated the binary in the trace message. </div></blockquote><blockquote style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;font-family:Helvetica, Arial, sans-serif;border-left-color:rgb(204,204,204);"><div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><div style="font-family:Helvetica, Arial, sans-serif;">Going through the binary vheap and getting a list of the processes that have references to those binaries won’t help in this case.</div><div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><div style="font-family:Helvetica, Arial, sans-serif;">Is such a trace possible? Is there some flag I can set when starting my BEAM process to give me some kind of debug output that would give me this information? I’d imagine this is all in C, so it might be a bit tricky...</div></blockquote><div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><div style="font-family:Helvetica, Arial, sans-serif;">No it is not possible without modifying the VM. I can't think of any good way to get this information without scanning the process' vheap, which as you say would not help much in this case.</div><div style="font-family:Helvetica, Arial, sans-serif;"></div><div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;font-family:Helvetica, Arial, sans-serif;border-left-color:rgb(204,204,204);"><div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><div style="font-family:Helvetica, Arial, sans-serif;">Thanks in advance for the help!</div><div style="font-family:Helvetica, Arial, sans-serif;"><br clear="none"></div><div style="font-family:Helvetica, Arial, sans-serif;">Cheers,</div><div style="font-family:Helvetica, Arial, sans-serif;">Devon</div>-- <br clear="none"><div dir="ltr" style="font-family:Helvetica, Arial, sans-serif;"><div dir="ltr" style="font-family:Helvetica, Arial, sans-serif;"><br clear="none">_________________<br clear="none">Devon Estes<br clear="none"><a rel="nofollow" shape="rect" target="_blank" onclick="return window.theMainWindow.showLinkWarning(this)" href="http://203.559.0323" style="font-family:Helvetica, Arial, sans-serif;">203.559.0323</a><br clear="none"><a rel="nofollow" shape="rect" target="_blank" href="http://www.devonestes.com" style="font-family:Helvetica, Arial, sans-serif;">www.devonestes.com</a><br clear="none"><br clear="none"></div></div>
</blockquote></div></div>
</blockquote></div></div></div></div></div>
</div>
</div></div></blockquote></div></div></div></div></div></div>
</div>
</div></body></html>