<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>All,<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>I have a TCP interface between an Erlang system and a C
system. Both send/receive marshaled binary Erlang terms and I have not
had any problems to date.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Today I began doing some more serious testing with larger
chunks of binary to be decoded in C.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>We ran into a bug (it seems) with erl_interface 3.5.5.4 that
is causing it to segfault during decoding. The backtrace looks like this:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Program received signal SIGSEGV, Segmentation fault.<o:p></o:p></p>
<p class=MsoNormal>[Switching to Thread 46912496233216 (LWP 4091)]<o:p></o:p></p>
<p class=MsoNormal>0x00000000004032c4 in _erl_free_term ()<o:p></o:p></p>
<p class=MsoNormal>(gdb) bt<o:p></o:p></p>
<p class=MsoNormal>#0 0x00000000004032c4 in _erl_free_term ()<o:p></o:p></p>
<p class=MsoNormal>#1 0x000000000040496b in erl_decode_it ()<o:p></o:p></p>
<p class=MsoNormal>#2 0x0000000000404937 in erl_decode_it ()<o:p></o:p></p>
<p class=MsoNormal>#3 0x0000000000404c93 in erl_decode_it ()<o:p></o:p></p>
<p class=MsoNormal>#4 0x0000000000404c93 in erl_decode_it ()<o:p></o:p></p>
<p class=MsoNormal>#5 0x0000000000405311 in erl_decode ()<o:p></o:p></p>
<p class=MsoNormal>#6 0x0000000000401b38 in main (argc=1,
argv=0x7fff51ec7938) at badtest.c:28<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>The unfortunate part is that the way this large binary term
is generated cannot be done in any kind of sample code (it’s being pulled off
an external database).<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>However, I have created a set of test files in C which
recreate the segfault. I stored the binary in a flat file (as
‘badbinary’) and have a testing program which reads it off disk and attempts to
decode it. To prove the approach is sane (and that this segfault is
related to something strange about the decoding of this particular binary, not
the size or general format of the binary) there is a ‘goodbinary’ file and
testing program for that.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>To use the test code:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Untar/Ungzip the file. You may need to edit the
Makefile to fix the paths to your erl_interface library.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>‘make’ and then you can:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>./badtest (this reads ‘badbinary’ and attempts to
decode, causes segfault)<o:p></o:p></p>
<p class=MsoNormal>./goodtest (this reads ‘goodbinary’ and successfully decodes
it) [nearly identical code to badtest.c but reads different file w/
different size]<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Also included is<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>./makegoodbin (a simple program that generates a large ETERM
in an identical format to the badbinary but contains duplicated binary data
everywhere) <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Notes:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>* The marshaled binary erlang term being sent to C can be
successfully decoded/unmarshaled from within Erlang without a problem<o:p></o:p></p>
<p class=MsoNormal>* This is reproducible with many different large erlang
terms generated from our database queries. ‘makegoodbin.c’ creates a term
identical in format to those causing problems, however it does not have the
random distribution of binary sizes and content, and so I’m not able to
reproduce the problem in this way.<o:p></o:p></p>
<p class=MsoNormal>* The entire system, end-to-end including this decoding
step, works perfectly in most cases. However when the data goes into the
100k+ range, the segfaults start to happen. That’s why I created the
‘makegoodbin’ which follows the same format. Unfortunately that works
even at sizes of >1MB adding to the confusion of the problem.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Any help is appreciated. Thanks.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Jonathan Gray<o:p></o:p></p>
<p class=MsoNormal>Streamy Inc.<o:p></o:p></p>
</div>
</body>
</html>