<div dir="ltr"><div><div><div><div>Hi Ross,<br><br></div>Wow, thanks for the extensive detective work! Once upon a time I was a full time C++ developer, only using Visual Studio and it's debugger. Gdb always looked so arcane in comparison, but this is an excellent reminder that there's no excuse for me to dig into gdb.<br><br></div>In the mean time it's confirmed that my issue is caused by a bug in enif_binary_to_term, but many thanks for you time anyway.<br><br></div>Best,<br></div>Vincent<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 15, 2018 at 5:01 PM, Ross Schlaikjer <span dir="ltr"><<a href="mailto:rossschlaikjer@gmail.com" target="_blank">rossschlaikjer@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Vincent,</div><div><br></div><div>I'm not entirely sure yet why this is happening, but I think I have at least figured out at least some of what is going on.</div><div><br></div><div>Heap allocation in erts is done very simply (for the most part), in that it is linearly allocated like so (as of otp git tag OTP-19.2.1): </div><div><br></div><div><font face="monospace, monospace"> 140 static ERTS_INLINE Eterm* alloc_heap(ErlNifEnv* env, size_t need)</font></div><div><font face="monospace, monospace"> 141 {</font></div><div><font face="monospace, monospace"> 142 Eterm* hp = env->hp;</font></div><div><font face="monospace, monospace"> 143 env->hp += need;</font></div><div><font face="monospace, monospace"> 144 if (env->hp <= env->hp_end) {</font></div><div><font face="monospace, monospace"> 145 return hp;</font></div><div><font face="monospace, monospace"> 146 }</font></div><div><font face="monospace, monospace"> 147 return alloc_heap_heavy(env, need, hp);</font></div><div><font face="monospace, monospace"> 148 }</font></div><div><br></div><div>When allocating a new cons cell, we take two 64bit words, store car/cdr and return the address of the base pointer + the list tag (0x1).</div><div><br></div><div><font face="monospace, monospace">1726 ERL_NIF_TERM enif_make_list_cell(ErlNifEnv* env, Eterm car, Eterm cdr)</font></div><div><font face="monospace, monospace">1727 {</font></div><div><font face="monospace, monospace">1728 Eterm* hp = alloc_heap(env,2);</font></div><div><font face="monospace, monospace">1729 Eterm ret = make_list(hp);</font></div><div><font face="monospace, monospace">1730 </font></div><div><font face="monospace, monospace">1731 ASSERT_IN_ENV(env, car, 0, "head of list cell");</font></div><div><font face="monospace, monospace">1732 ASSERT_IN_ENV(env, cdr, 0, "tail of list cell");</font></div><div><font face="monospace, monospace">1733 CAR(hp) = car;</font></div><div><font face="monospace, monospace">1734 CDR(hp) = cdr;</font></div><div><font face="monospace, monospace">1735 return ret;</font></div><div><font face="monospace, monospace">1736 } </font></div><div><br></div><div><font face="monospace, monospace">make_list</font> above epands to the macro</div><div><font face="monospace, monospace">#define _unchecked_make_list(x) ((Uint)(x) + TAG_PRIMARY_LIST)</font></div><div>so all it's doing is adding 1 to that ret pointer.</div><div><br></div><div>If we take this all to gdb, we can step through your code and see the list modifications occurring (comments inline):</div><div><br></div><div><font face="monospace, monospace">18<span style="white-space:pre-wrap"> </span> ERL_NIF_TERM list = enif_make_list(env, 0);</font></div><div><font face="monospace, monospace">(gdb) n</font></div><div><font face="monospace, monospace">19<span style="white-space:pre-wrap"> </span> ERL_NIF_TERM in_term = enif_make_uint(env, 42);</font></div><div><font face="monospace, monospace">(gdb) p/x list</font></div><div><font face="monospace, monospace">$1 = 0xfffffffffffffffb # This is the 'empty list' value</font></div><div><font face="monospace, monospace">(gdb) n</font></div><div><font face="monospace, monospace">22<span style="white-space:pre-wrap"> </span> int success = enif_term_to_binary(env, in_term, &bin);</font></div><div><font face="monospace, monospace">(gdb) n</font></div><div><font face="monospace, monospace">24<span style="white-space:pre-wrap"> </span> int bcount = enif_binary_to_term(env, bin.data, 3, &out_term1, 0);</font></div><div><font face="monospace, monospace">(gdb) p/x env->hp # Check the heap pointer before we call enif_binary_to_term</font></div><div><font face="monospace, monospace">$4 = 0x7fff79840fe0</font></div><div><font face="monospace, monospace">(gdb) n</font></div><div><font face="monospace, monospace">25<span style="white-space:pre-wrap"> </span> list = enif_make_list_cell(env, out_term1, list);</font></div><div><font face="monospace, monospace">(gdb) p/x env->hp # Check again after - notice the hp hasn't changed</font></div><div><font face="monospace, monospace">$5 = 0x7fff79840fe0</font></div><div><font face="monospace, monospace">(gdb) n # Make the cons cell</font></div><div><font face="monospace, monospace">27<span style="white-space:pre-wrap"> </span> bcount = enif_binary_to_term(env, bin.data, 3, &out_term2, 0);</font></div><div><font face="monospace, monospace">(gdb) p/x list # As we can see, it has the address of the base pointer + the list tag (1)</font></div><div><font face="monospace, monospace">$6 = 0x7fff79840fe1</font></div><div><font face="monospace, monospace">(gdb) x/2g list-1 # If we examine it, we see it has the head (our immediate int value) and the tail (empty list)</font></div><div><font face="monospace, monospace">0x7fff79840fe0:<span style="white-space:pre-wrap"> </span>0x00000000000002af<span style="white-space:pre-wrap"> </span>0xfffffffffffffffb</font></div><div><font face="monospace, monospace">(gdb) p/x env->hp # Our heap pointer has gone up by 16 bytes, as expected (2 64bit words)</font></div><div><font face="monospace, monospace">$7 = 0x7fff79840ff0</font></div><div><font face="monospace, monospace">(gdb) n # Now we execute the second enif_binary_to_term call</font></div><div><font face="monospace, monospace">28<span style="white-space:pre-wrap"> </span> list = enif_make_list_cell(env, out_term2, list);</font></div><div><font face="monospace, monospace">(gdb) p/x env->hp</font></div><div><font face="monospace, monospace">$8 = 0x7fff79840fe0 # This is where our problem begins! binary_to_term has _decremented_ the heap pointer!</font></div><div><font face="monospace, monospace">(gdb) n # Create our second cons cell</font></div><div><font face="monospace, monospace">30<span style="white-space:pre-wrap"> </span> bcount = enif_binary_to_term(env, bin.data, 3, &out_term3, 0);</font></div><div><font face="monospace, monospace">(gdb) p/x list # We look at the 'new' list pointer value... but it's the same as before!</font></div><div><font face="monospace, monospace">$9 = 0x7fff79840fe1</font></div><div><font face="monospace, monospace">(gdb) x/2g list -1</font></div><div><font face="monospace, monospace">0x7fff79840fe0:<span style="white-space:pre-wrap"> </span>0x00000000000002af<span style="white-space:pre-wrap"> </span>0x00007fff79840fe1 # This list now references itself!</font></div><div><br></div><div>It looks like when we call binary_to_term, the cache_env call is what's messing with our heap pointer -</div><div>(gdb) watch env->hp</div><div><font face="monospace, monospace">Hardware watchpoint 2: env->hp</font></div><div><font face="monospace, monospace">(gdb) n</font></div><div><font face="monospace, monospace"><br></font></div><div><font face="monospace, monospace">Thread 14 "1_scheduler" hit Hardware watchpoint 2: env->hp</font></div><div><font face="monospace, monospace"><br></font></div><div><font face="monospace, monospace">Old value = (Eterm *) 0x7fff79840ff0</font></div><div><font face="monospace, monospace">New value = (Eterm *) 0x7fff79840fe0</font></div><div><font face="monospace, monospace">0x00005555556ce798 in cache_env (env=0x7fffb34bee40) at beam/erl_nif.c:421</font></div><div><font face="monospace, monospace">warning: Source file is more recent than executable.</font></div><div><font face="monospace, monospace">421<span style="white-space:pre-wrap"> </span>env->hp = HEAP_TOP(env->proc);</font></div><div><font face="monospace, monospace">(gdb) bt</font></div><div><font face="monospace, monospace">#0 0x00005555556ce798 in cache_env (env=0x7fffb34bee40) at beam/erl_nif.c:421</font></div><div><font face="monospace, monospace">#1 enif_binary_to_term (dst_env=0x7fffb34bee40, data=0x7fffb5b82b68 "\203a*", data_sz=<optimized out>, term=0x7fffb34bed28, </font></div><div><font face="monospace, monospace"> opts=(unknown: 0)) at beam/erl_nif.c:1161</font></div><div><br></div><div><br></div><div>So as far as I can tell, something to do with the switchover between HAlloc and alloc_heap (flush_env / cache_env) seems to be losing some bytes off the heap pointer, but _only_ for subsequent calls. Unfortunately I must go back to my day job for now, but will try and debug more later.</div><div><br></div><div>Hope this helps.</div><div><br></div><div>- Ross S</div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">On Mon, May 14, 2018 at 4:57 PM, Vincent Siliakus <span dir="ltr"><<a href="mailto:zambal@gmail.com" target="_blank">zambal@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="h5"><div dir="ltr"><div>Hi all,</div><div><br></div><div>I'm writing a NIF library and can't wrap my head around why the following code makes the erlang runtime hang when called from a shell:</div><div><br></div><div><font face="monospace,monospace"> static ERL_NIF_TERM test(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) {<br> ErlNifBinary bin;<br><br> ERL_NIF_TERM list = enif_make_list(env, 0);<br> ERL_NIF_TERM in_term = enif_make_uint(env, 42);<br> ERL_NIF_TERM out_term1, out_term2, out_term3;<br><br> enif_term_to_binary(env, in_term, &bin);<br><br> enif_binary_to_term(env, bin.data, bin.size, &out_term1, 0);<br> list = enif_make_list_cell(env, out_term1, list);<br><br> enif_binary_to_term(env, bin.data, bin.size, &out_term2, 0);<br> list = enif_make_list_cell(env, out_term2, list);<br><br> enif_binary_to_term(env, bin.data, bin.size, &out_term3, 0);<br> list = enif_make_list_cell(env, out_term3, list);<br><br> return list;<br> }</font></div><div><font face="monospace,monospace"><br></font></div><div>The <span style="font-family:arial,helvetica,sans-serif"></span><font face="monospace,monospace"><span style="font-family:arial,helvetica,sans-serif">multiple calls to </span>enif_binary_to_term</font> somehow <span style="font-family:arial,helvetica,sans-serif">seem to corrupt memory in the calling environment, so I'm probably using it incorrectly. Could some kind soul point me to the error? I'm running this code on OTP20 / erts-9.2.<br></span></div><div><span style="font-family:arial,helvetica,sans-serif"><br></span></div><div><span style="font-family:arial,helvetica,sans-serif">Thanks in advance,</span></div><div><span style="font-family:arial,helvetica,sans-serif">Vincent<br></span></div></div>
<br></div></div>______________________________<wbr>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/list<wbr>info/erlang-questions</a><br>
<br></blockquote></div><br></div></div>
</blockquote></div><br></div>