<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>All,<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I have a TCP interface between an Erlang system and a C
system. Both send/receive marshaled binary Erlang terms and I have not
had any problems to date.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Today I began doing some more serious testing with larger chunks
of binary to be decoded in C.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>We ran into a bug (it seems) with erl_interface 3.5.5.4 that
is causing it to segfault during decoding. The backtrace looks like this:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Program received signal SIGSEGV, Segmentation fault.<o:p></o:p></p>
<p class=MsoNormal>[Switching to Thread 46912496233216 (LWP 4091)]<o:p></o:p></p>
<p class=MsoNormal>0x00000000004032c4 in _erl_free_term ()<o:p></o:p></p>
<p class=MsoNormal>(gdb) bt<o:p></o:p></p>
<p class=MsoNormal>#0 0x00000000004032c4 in _erl_free_term ()<o:p></o:p></p>
<p class=MsoNormal>#1 0x000000000040496b in erl_decode_it ()<o:p></o:p></p>
<p class=MsoNormal>#2 0x0000000000404937 in erl_decode_it ()<o:p></o:p></p>
<p class=MsoNormal>#3 0x0000000000404c93 in erl_decode_it ()<o:p></o:p></p>
<p class=MsoNormal>#4 0x0000000000404c93 in erl_decode_it ()<o:p></o:p></p>
<p class=MsoNormal>#5 0x0000000000405311 in erl_decode ()<o:p></o:p></p>
<p class=MsoNormal>#6 0x0000000000401b38 in main (argc=1,
argv=0x7fff51ec7938) at badtest.c:28<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>The unfortunate part is that the way this large binary term
is generated cannot be done in any kind of sample code (it’s being pulled
off an external database).<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>However, I have created a set of test files in C which
recreate the segfault. I stored the binary in a flat file (as ‘badbinary’)
and have a testing program which reads it off disk and attempts to decode
it. To prove the approach is sane (and that this segfault is related to
something strange about the decoding of this particular binary, not the size or
general format of the binary) there is a ‘goodbinary’ file and
testing program for that.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>To use the test code:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Untar/Ungzip the file. You may need to edit the
Makefile to fix the paths to your erl_interface library.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>‘make’ and then you can:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>./badtest (this reads ‘badbinary’ and
attempts to decode, causes segfault)<o:p></o:p></p>
<p class=MsoNormal>./goodtest (this reads ‘goodbinary’ and
successfully decodes it) [nearly identical code to badtest.c but reads
different file w/ different size]<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Also included is<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>./makegoodbin (a simple program that generates a large ETERM
in an identical format to the badbinary but contains duplicated binary data everywhere)
<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Notes:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>* The marshaled binary erlang term being sent to C can be
successfully decoded/unmarshaled from within Erlang without a problem<o:p></o:p></p>
<p class=MsoNormal>* This is reproducible with many different large erlang
terms generated from our database queries. ‘makegoodbin.c’ creates
a term identical in format to those causing problems, however it does not have the
random distribution of binary sizes and content, and so I’m not able to
reproduce the problem in this way.<o:p></o:p></p>
<p class=MsoNormal>* The entire system, end-to-end including this decoding
step, works perfectly in most cases. However when the data goes into the
100k+ range, the segfaults start to happen. That’s why I created
the ‘makegoodbin’ which follows the same format.
Unfortunately that works even at sizes of >1MB adding to the confusion of
the problem.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Any help is appreciated. Thanks.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Jonathan Gray<o:p></o:p></p>
<p class=MsoNormal>Streamy Inc.<o:p></o:p></p>
</div>
</body>
</html>