<div>
Paul / Kostis / Group
</div><div><br></div><div>First off many thanks for your help - I have got to the bottom of what was happening. Basically there are beam files included in the src release tar balls that meant certain beams were not being built. The details are below and I would be interested to know if the presence of the beam files in the source tar balls is deliberate.</div><div><br></div><div>I was building Erlang from a script that did the following:</div><div><br></div><div><pre style="box-sizing: border-box; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; margin-top: 0px; margin-bottom: 0px; color: rgb(51, 51, 51); line-height: 18px;"><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;"><span class="c" style="box-sizing: border-box; color: rgb(153, 153, 136); font-style: italic;"># Download and install erlang</span></div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;"><span class="nv" style="box-sizing: border-box; color: teal;">OtpDir</span><span class="o" style="box-sizing: border-box; font-weight: bold;">=</span>otp_src_R16B03-1</div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;"><br style="box-sizing: border-box;"></div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;">rm -rf <span class="nv" style="box-sizing: border-box; color: teal;">$OtpDir</span></div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;"><br style="box-sizing: border-box;"></div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;">wget http://www.erlang.org/download/otp_src_R16B03-1.tar.gz</div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;">tar xf otp_src_R16B03-1.tar.gz</div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;">rm otp_src_R16B03-1.tar.gz</div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;"><br style="box-sizing: border-box;"></div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;"><span class="nb" style="box-sizing: border-box; color: rgb(0, 134, 179);">pushd</span> <span class="nv" style="box-sizing: border-box; color: teal;">$OtpDir</span></div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;"><br style="box-sizing: border-box;"></div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;">./configure --enable-native-libs</div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;">make</div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;">sudo make install</div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;"><span class="nb" style="box-sizing: border-box; color: rgb(0, 134, 179);">popd</span></div></pre></div>
<div><div><br></div><div>Every time it tried again I would re-run the script so that what I was doing was entirely reproducible.</div><div><br></div><div>The issue is that the <span style="white-space: pre-wrap; color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; line-height: 18px;">otp_src_R16B03-1.tar.gz </span>contains files in <font face="Courier New">otp_src_R16B03-1/lib/stdlib/ebin/</font> including <font face="Courier New">lists.beam</font> and others and the make process above was not marking the beams in that directory as obsolete (the beam files don't have a dependency on the Makefile). I was concerned about side-effects from previous attempts impacting settings so between every attempt ran "<font color="#333333" face="Consolas, Liberation Mono, Courier, monospace"><span style="font-size: 12px; line-height: 18px; white-space: pre-wrap;">rm -rf otp_src_R16B03-1”</span></font> and started again - reintroducing the obsolete beams every time.</div><div><br></div><div>I think the minimal set of commands to get a native build from the src release tar is:</div><div><br></div><div><pre style="box-sizing: border-box; margin-top: 0px; margin-bottom: 0px; color: rgb(51, 51, 51); line-height: 18px;"><div class="line" style="font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; box-sizing: border-box; padding-left: 10px; height: 18px;">./configure --enable-native-libs</div><div class="line" style="font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; box-sizing: border-box; padding-left: 10px; height: 18px;">make clean # you need to run ./configure before clean as you don’t have a Makefile otherwise</div><div class="line" style="font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; box-sizing: border-box; padding-left: 10px; height: 18px;">./configure --enable-native-libs # as the make clean will have deleted the settings</div><div class="line" style="box-sizing: border-box; padding-left: 10px; height: 18px;"><font face="Helvetica">make</font></div><div class="line" style="font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; box-sizing: border-box; padding-left: 10px; height: 18px;">sudo make install</div></pre></div><div><br></div><div>This deletes the beams included in the top tar ball and then runs make from a clean base. Pulling the source from Github should work as well as the beams are not part of the repo. </div><div><br></div><div>Thanks again for your help,</div><div><br></div><div>Adrian</div><div>-- </div><div>Adrian Roe</div><div>Sent with <a href="http://www.sparrowmailapp.com/?sig">Sparrow</a></div><div><br></div><div><span style="color: rgb(160, 160, 168);">On Wednesday, 2 April 2014 at 21:31, Kostis Sagonas wrote:</span></div></div>
<blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px;">
<span><div><div><div>On 04/02/2014 08:11 PM, Adrian Roe wrote:</div><blockquote type="cite"><div><div>We’ve been using Erlang in production for quite a few years now and for</div><div>the first time we have a project that really benefits from HIPE / native</div><div>code being turned on (before things have always been fast enough without</div><div>and the benefit of more detailed stack traces etc outweighed any</div><div>performance gains). We did some benchmarking by compiling single</div><div>modules with HIPE - both in our application code and then going through</div><div>key OTP / Erlang library modules, clearing sticky_directory errors and</div><div>recompiling them one by one with +native.</div><div><br></div><div>Now that it is clear that (for this use case) native / HIPE is the way</div><div>to go, I went to the project’s build box and recompiled Erlang (R16B03)</div><div>from source with</div><div><br></div><div>./configure --enable-hipe --enable-native-libs</div><div>in the expectation that all libraries would be compiled with +native</div><div><br></div><div>I’ve tried this on OSX, CentOS and Ubuntu all with the same result, so</div><div>it’s clearly me doing something stupid, but I can’t see it.</div></div></blockquote><div><br></div><div>Can it be that you have not done a 'make clean'? Note that ./configure </div><div>(with or without flags) does not actually build the system or recreate </div><div>any dependencies.</div><div><br></div><div>Can you try the following sequence of commands on your Ubuntu machine </div><div>(unfortunately, I've never tried HiPE on OSX or CentOS to know what is </div><div>its status there) and tell us what happened?</div><div><br></div><div> make clean</div><div> ./otp_build autoconf</div><div> ./configure --enable-native-libs</div><div> make -j</div><div><br></div><div>As Sverker suggested, you can find out whether some module, say lists, </div><div>contains native code or not by: code:is_module_native(lists).</div><div><br></div><div><br></div><div>Some more info:</div><div><br></div><div> - Note that for better or worse, --enable-native-libs does not build </div><div>the complete set of libraries with native code. Only kernel, stdlib, </div><div>compiler, hipe, syntax_tools and dialyzer are built with native code. If </div><div>your application requires some other modules in native code, you need to </div><div>explicitly compile them to native or change the Makefile of their </div><div>library in a way similar to the Makefiles of these libraries.</div><div><br></div><div> - I would recommend against R16B03 as it is seriously broken in some </div><div>respects. (This comment is unrelated to native code.) Use R16B03-1 </div><div>instead.</div><div><br></div><div>Kostis</div></div></div></span>
</blockquote>
<div>
<br>
</div>