From mj@REDACTED  Tue Jul  1 09:13:34 2008
From: mj@REDACTED (Mikkel Jensen)
Date: Tue, 1 Jul 2008 09:13:34 +0200
Subject: [erlang-bugs] Bug in xmerl
In-Reply-To: <b852c190806270557u20f76c5p8bbd6e1cbe50120e@mail.gmail.com>
References: <b852c190806270557u20f76c5p8bbd6e1cbe50120e@mail.gmail.com>
Message-ID: <b852c190807010013m30123f84tf680eef28b20e56d@mail.gmail.com>

Is it possible for someone from the OTP team to confirm if this is a bug or
not?

If it is I could really use a patch :-)

- Mikkel

On Fri, Jun 27, 2008 at 2:57 PM, Mikkel Jensen <mj@REDACTED> wrote:

> It seems there is a bug in xmerl when loading elements that contain numeric
> character references followed by UTF-8 characters.
>
> Example: ? newline ?
>
> 1> element(1, xmerl_scan:string("<a>\303\251&#xD;\303\251</a>", [{encoding,
> 'utf-8'}])).
> {xmlElement,a,a,[],
>             {xmlNamespace,[],[]},
>             [],1,[],
>             [{xmlText,[{a,1}],1,[],"\303\251",text},
>              {xmlText,[{a,1}],2,[],[10,195,131,194,169],text}],
>             [],"/",undeclared}
>
> Xmerl splits the parsed value around the newline character (strange but
> ok). However, the first part is encoded correctly while the second part is
> garbled!
>
> It's worth noticing that attribute values are encoded correctly:
>
> 2> element(1, xmerl_scan:string("<a b=\"\303\251&#xD;\303\251\"/>",
> [{encoding, 'utf-8'}])).
> {xmlElement,a,a,[],
>             {xmlNamespace,[],[]},
>             [],1,
>             [{xmlAttribute,b,[],[],[],[],1,[],"\303\251 \303\251",false}],
>             [],[],"/",undeclared}
>
> - Mikkel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080701/6123c921/attachment.htm>

From erlang-questions_efine@REDACTED  Wed Jul  2 01:06:52 2008
From: erlang-questions_efine@REDACTED (Edwin Fine)
Date: Tue, 1 Jul 2008 19:06:52 -0400
Subject: [erlang-bugs] [BUG] OTP R12B-3: HTTP "space after headers" bug in
	inet_drv.c
Message-ID: <6c2563b20807011606x7f00fa5bne75649182294bfba@mail.gmail.com>

First of all, many thanks to Claes Wikstrom for isolating and finding the
module in which this bug lives (and for doing it so quickly). I originally
thought this was a Yaws bug, and emailed Claes with the symptoms. It's quite
repeatable (test code at end of email). He traced the bug to inet_drv.c and
emailed me back with a small test program and details, and I offered to
report the bug. Then I got curious as to what it could have been, so I did
some debugging.

The bug is in inet_drv.c. It will cause the end of the HTTP headers to be
missed if the first character following the last CRLF (the one on its "own
line") is a space or tab. This causes Yaws to time out after 30 seconds
waiting for more header data that will never arrive. I am sure this will be
true for httpd, too.

The culprit (well, sort of; if you read on you will see why) is the
following code on line 8324:

*if (SP(ptr2+1)) {*
    ptr1 = ptr2+1;
    len = n - plen;
}
else
    goto done;

Consider the following HTTP POST data:

"POST /invalid/url HTTP/1.1\r\n"
"Connection: close\r\n"
"Host: localhost:8000\r\n"
"User-Agent: perl post\r\n"
"Content-Length: 4\r\n"
"Content-Type: text/xml; charset=utf-8\r\n"
"\r\n"
" postdata..." <-- Note this data starts with a space character

When the inet_drv.c code gets to the last CRLF, it doesn't detect that it's
the last CRLF (end of headers) and checks to see if the next character is a
space or tab using the macro SP(ptr2+1). If there's any data following the
last CRLF and it starts with a space or tab, the code as it stands now will
think it's another header line and try to get more data, then time out when
nothing arrives.

What needed to be done was to check to see if the data (in that particular
state) was an LF or CRLF standing on its own, and only check for a space
following that if not. This check correctly detects the CRLF that terminates
the header data. I added two lines of code to inet_drv.c to fix that error,
and the Erlang test code (kindly supplied by Claes, and at the end of this
email) now runs without error. I can't guarantee that the fix will work
under all circumstances, and it needs to be tested thoroughly, but it's a
starting point and illustrates the problem. (I would say that the HTTP
header parsing in inet_drv.c probably could be beefed up a little).

The patch to inet_drv.c is as follows (tabs may not be correct because I
reformatted the code):

8318a8319,8323
>
> /* Test needed in case buffer is in form "\r\n\srequestdata" where \s is
SP or TAB */
> if (((plen == 1) && NL(ptr)) || ((plen == 2) && CRNL(ptr)))
>     goto done;
>

Claes's test code is here (slightly modified by me to make it easier to test
the normal case and the "bug" case). You can run it as
post:p(Host,Port,data) or post:p(Host,Port,bug).

Regards,
Edwin Fine
======================================
-module(post).
-compile(export_all).

p() ->
    p({127,0,0,1},8000,data).

p(Host, Port, What) when What =:= bug; What =:= data ->
   {ok, S} = gen_tcp:connect(Host, Port, [{active, true}]),
   gen_tcp:send(S, select_data(What)),
   recloop().

recloop() ->
   receive
       X ->
           io:format("GOT ~p~n", [X]),
           recloop()
   after 4000 ->
           timeout
   end.

select_data(What) ->
    case What of
        bug ->
            bug();
        data ->
            data()
    end.

data() ->
   Msg = "dfoo",
   H = "POST /invalid/url HTTP/1.1\r\n"
       "Connection: close\r\n"
       "Host: localhost:8000\r\n"
       "User-Agent: perl post\r\n"
       "Content-Length: 4\r\n"
       "Content-Type: text/xml; charset=utf-8\r\n"
       "\r\n",
   H ++ Msg.


%% space bug
bug() ->
   Msg = " foo",
   H = "POST /invalid/url HTTP/1.1\r\n"
       "Connection: close\r\n"
       "Host: localhost:8000\r\n"
       "User-Agent: perl post\r\n"
       "Content-Length: 4\r\n"
       "Content-Type: text/xml; charset=utf-8\r\n"
       "\r\n",
   H ++ Msg.



-- 
The great enemy of the truth is very often not the lie -- deliberate,
contrived and dishonest, but the myth, persistent, persuasive, and
unrealistic. Belief in myths allows the comfort of opinion without the
discomfort of thought.
John F. Kennedy 35th president of US 1961-1963 (1917 - 1963)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080701/97264cb6/attachment.htm>

From pfisher@REDACTED  Wed Jul  2 02:24:05 2008
From: pfisher@REDACTED (Paul Fisher)
Date: Tue, 01 Jul 2008 19:24:05 -0500
Subject: [erlang-bugs] erts_port[].drv_ptr == 0,
	when erts_port[].status not free
Message-ID: <1214958245.16472.46.camel@localhost>

We have a system where we run lots of linked-in driver ports that get
created/used/closed frequently and sometimes very quickly.  Today when
several open_port/2, port_command/2 and port_close/1 cycles happened
rapid succession, a SIGSEGV occurrect in erl_bif_ddl.c:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1125235040 (LWP 12087)]
0x0000000000449712 in erl_ddll_try_unload_2 (p=0x2aaaab11fc90,
    name_term=659339, options=46912503328425) at beam/erl_bif_ddll.c:592

The emulator was run on a Q6600 (quad-core, 2.4Ghz), and started with +A
8,
and the linked-in driver executes the bulk of its work with
driver_async().
There were continuously 8 driver cycles running for 5-10 seconds before
the
segfault occurred.

?(gdb) where
#0  0x0000000000449712 in erl_ddll_try_unload_2 (p=0x2aaaab11fc90,
    name_term=659339, options=46912503328425) at beam/erl_bif_ddll.c:592
#1  0x000000000052337f in process_main () at beam/beam_emu.c:2073
#2  0x000000000049c213 in sched_thread_func (vesdp=0x2ae18cb74f98)
    at beam/erl_process.c:741
#3  0x00000000005b6818 in thr_wrapper (vtwd=0x7fff1eb77de0)
    at common/ethread.c:474
#4  0x00002ae18c530f1a in start_thread () from /lib/libpthread.so.0
#5  0x00002ae18c8135d2 in clone () from /lib/libc.so.6
#6  0x0000000000000000 in ?? ()

So the code at the point of the SIGSEGV @ erl_bif_ddll.c:592 says:

        for (j = 0; j < erts_max_ports; j++) {
=>          if (!(erts_port[j].status &  FREE_PORT_FLAGS)
                && erts_port[j].drv_ptr->handle == dh) {

It appears that the code assumes that if the erts_port array entry being
evaluated during the search has a valid (non-zero) drv_ptr value, if the
entry is not marked as free.  At the time of the crash, this is clearly
not
the case:

(gdb) p j
$8 = 896

(gdb) p erts_port[j]
$7 = {sched = {next = 0x0, prev = 0x0, taskq = 0x0, exe_taskq = 0x0},
  timeout_task = {counter = 0}, refc = {counter = 2}, lock = 0x81b3c8,
  xports = 0x0, id = 14343, connected = 0, caller = 0, data = 0, bp =
0x0,
  nlinks = 0x0, monitors = 0x0, bytes_in = 0, bytes_out = 0, ptimer =
0x0,
  tracer_proc = 18446744073709551611, trace_flags = 0, ioq = {size = 0,
    v_start = 0x0, v_end = 0x0, v_head = 0x0, v_tail = 0x0, v_small = {{
        iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {
        iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {
        iov_base = 0x0, iov_len = 0}}, b_start = 0x0, b_end = 0x0,
    b_head = 0x0, b_tail = 0x0, b_small = {0x0, 0x0, 0x0, 0x0, 0x0}},
  dist_entry = 0x0, name = 0x0, drv_ptr = 0x0, drv_data = 0, suspended =
0x0,
  linebuf = 0x0, status = 4096, control_flags = 0, reg = 0x0,
  port_data_lock = 0x0}

(gdb) p erts_port[j].drv_ptr
$6 = (ErlDrvEntry *) 0x0


So the real questions are: 1) is whether the assumption built into this
code is correct; and 2) if so, how did we get in the position of
violating
it.  I'd appreciate some insight into what could be going on here, and
where I should can start looking.


-- 
paul



From lars@REDACTED  Wed Jul  2 09:38:58 2008
From: lars@REDACTED (Lars Thorsen)
Date: Wed, 02 Jul 2008 09:38:58 +0200
Subject: [erlang-bugs] Bug in xmerl
In-Reply-To: <b852c190807010013m30123f84tf680eef28b20e56d@mail.gmail.com>
References: <b852c190806270557u20f76c5p8bbd6e1cbe50120e@mail.gmail.com>
	<b852c190807010013m30123f84tf680eef28b20e56d@mail.gmail.com>
Message-ID: <486B3092.8060604@erix.ericsson.se>


Hi,
it was a bug in xmerl. The ending parenthesis in the call to 
string_to_char_set/2 (line 2449 in xmerl_scan)was placed wrong.
This will be fixed in R12B-4 but I include some patch lines below.

------------------------- Patch start ----------------------------------
--- xmerl_scan.erl@@/main/xmerl/108     2008-04-25 09:20:41.000000000 +0200
+++ xmerl_scan.erl      2008-07-01 17:11:18.000000000 +0200
@@ -2446,7 +2446,7 @@
      case markup_delimeter(ExpRef) of
         true -> 
scan_content(ExpRef++T1,S1,Pos,Name,Attrs,Space,Lang,Parents,NS,Acc,ExpRef);
         _ ->
- 
scan_content(string_to_char_set(S1#xmerl_scanner.encoding,ExpRef++T1),S1,Pos,Name,Attrs,Space,Lang,Parents,NS,Acc,[])
+ 
scan_content(string_to_char_set(S1#xmerl_scanner.encoding,ExpRef)++T1,S1,Pos,Name,Attrs,Space,Lang,Parents,NS,Acc,[])
      end;
  scan_content("<!--" ++ T, S, Pos, Name, Attrs, Space, Lang, Parents, 
NS, Acc,[]) ->
      {_, T1, S1} = scan_comment(T, S, Pos, Parents, Lang),
------------------------- Patch end ----------------------------------

Regards Lars




Mikkel Jensen wrote:
> Is it possible for someone from the OTP team to confirm if this is a bug 
> or not?
> 
> If it is I could really use a patch :-)
> 
> - Mikkel
> 
> On Fri, Jun 27, 2008 at 2:57 PM, Mikkel Jensen <mj@REDACTED 
> <mailto:mj@REDACTED>> wrote:
> 
>     It seems there is a bug in xmerl when loading elements that contain
>     numeric character references followed by UTF-8 characters.
> 
>     Example: ? newline ?
> 
>     1> element(1, xmerl_scan:string("<a>\303\251&#xD;\303\251</a>",
>     [{encoding, 'utf-8'}])).
>     {xmlElement,a,a,[],
>                 {xmlNamespace,[],[]},
>                 [],1,[],
>                 [{xmlText,[{a,1}],1,[],"\303\251",text},
>                  {xmlText,[{a,1}],2,[],[10,195,131,194,169],text}],
>                 [],"/",undeclared}
> 
>     Xmerl splits the parsed value around the newline character (strange
>     but ok). However, the first part is encoded correctly while the
>     second part is garbled!
> 
>     It's worth noticing that attribute values are encoded correctly:
> 
>     2> element(1, xmerl_scan:string("<a b=\"\303\251&#xD;\303\251\"/>",
>     [{encoding, 'utf-8'}])).
>     {xmlElement,a,a,[],
>                 {xmlNamespace,[],[]},
>                 [],1,
>                 [{xmlAttribute,b,[],[],[],[],1,[],"\303\251
>     \303\251",false}],
>                 [],[],"/",undeclared}
> 
>     - Mikkel
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs



From raimo+erlang-bugs@REDACTED  Thu Jul  3 17:59:57 2008
From: raimo+erlang-bugs@REDACTED (Raimo Niskanen)
Date: Thu, 3 Jul 2008 17:59:57 +0200
Subject: [erlang-bugs] erts_port[].drv_ptr == 0,
	when erts_port[].status not free
In-Reply-To: <1214958245.16472.46.camel@localhost>
References: <1214958245.16472.46.camel@localhost>
Message-ID: <20080703155957.GB5148@erix.ericsson.se>

Thank you for your bug report.

We will look into your problem when the concerned developers
comes in after their vacation.

Can you give us host OS and Erlang release too?



On Tue, Jul 01, 2008 at 07:24:05PM -0500, Paul Fisher wrote:
> We have a system where we run lots of linked-in driver ports that get
> created/used/closed frequently and sometimes very quickly.  Today when
> several open_port/2, port_command/2 and port_close/1 cycles happened
> rapid succession, a SIGSEGV occurrect in erl_bif_ddl.c:
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 1125235040 (LWP 12087)]
> 0x0000000000449712 in erl_ddll_try_unload_2 (p=0x2aaaab11fc90,
>     name_term=659339, options=46912503328425) at beam/erl_bif_ddll.c:592
> 
> The emulator was run on a Q6600 (quad-core, 2.4Ghz), and started with +A
> 8,
> and the linked-in driver executes the bulk of its work with
> driver_async().
> There were continuously 8 driver cycles running for 5-10 seconds before
> the
> segfault occurred.
> 
> ???(gdb) where
> #0  0x0000000000449712 in erl_ddll_try_unload_2 (p=0x2aaaab11fc90,
>     name_term=659339, options=46912503328425) at beam/erl_bif_ddll.c:592
> #1  0x000000000052337f in process_main () at beam/beam_emu.c:2073
> #2  0x000000000049c213 in sched_thread_func (vesdp=0x2ae18cb74f98)
>     at beam/erl_process.c:741
> #3  0x00000000005b6818 in thr_wrapper (vtwd=0x7fff1eb77de0)
>     at common/ethread.c:474
> #4  0x00002ae18c530f1a in start_thread () from /lib/libpthread.so.0
> #5  0x00002ae18c8135d2 in clone () from /lib/libc.so.6
> #6  0x0000000000000000 in ?? ()
> 
> So the code at the point of the SIGSEGV @ erl_bif_ddll.c:592 says:
> 
>         for (j = 0; j < erts_max_ports; j++) {
> =>          if (!(erts_port[j].status &  FREE_PORT_FLAGS)
>                 && erts_port[j].drv_ptr->handle == dh) {
> 
> It appears that the code assumes that if the erts_port array entry being
> evaluated during the search has a valid (non-zero) drv_ptr value, if the
> entry is not marked as free.  At the time of the crash, this is clearly
> not
> the case:
> 
> (gdb) p j
> $8 = 896
> 
> (gdb) p erts_port[j]
> $7 = {sched = {next = 0x0, prev = 0x0, taskq = 0x0, exe_taskq = 0x0},
>   timeout_task = {counter = 0}, refc = {counter = 2}, lock = 0x81b3c8,
>   xports = 0x0, id = 14343, connected = 0, caller = 0, data = 0, bp =
> 0x0,
>   nlinks = 0x0, monitors = 0x0, bytes_in = 0, bytes_out = 0, ptimer =
> 0x0,
>   tracer_proc = 18446744073709551611, trace_flags = 0, ioq = {size = 0,
>     v_start = 0x0, v_end = 0x0, v_head = 0x0, v_tail = 0x0, v_small = {{
>         iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {
>         iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {
>         iov_base = 0x0, iov_len = 0}}, b_start = 0x0, b_end = 0x0,
>     b_head = 0x0, b_tail = 0x0, b_small = {0x0, 0x0, 0x0, 0x0, 0x0}},
>   dist_entry = 0x0, name = 0x0, drv_ptr = 0x0, drv_data = 0, suspended =
> 0x0,
>   linebuf = 0x0, status = 4096, control_flags = 0, reg = 0x0,
>   port_data_lock = 0x0}
> 
> (gdb) p erts_port[j].drv_ptr
> $6 = (ErlDrvEntry *) 0x0
> 
> 
> So the real questions are: 1) is whether the assumption built into this
> code is correct; and 2) if so, how did we get in the position of
> violating
> it.  I'd appreciate some insight into what could be going on here, and
> where I should can start looking.
> 
> 
> -- 
> paul
> 
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB


From mikage@REDACTED  Fri Jul 11 04:29:30 2008
From: mikage@REDACTED (Mikage Sawatari)
Date: Fri, 11 Jul 2008 11:29:30 +0900
Subject: [erlang-bugs] ssh_cm:open_pty bug
Message-ID: <5e448700807101929s77a4369ax65d7bffd6621e3ef@mail.gmail.com>

Hello,

ssh_cm:open_pty has a bug.
The attached patch corrects this problem.

-- 
-----------------------------------------------------------------------
SAWATARI Mikage (SANO Taku)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ssh_cm.patch
Type: application/octet-stream
Size: 889 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080711/e0c3f2f3/attachment.obj>

From nick@REDACTED  Fri Jul 11 10:43:20 2008
From: nick@REDACTED (Niclas Eklund)
Date: Fri, 11 Jul 2008 10:43:20 +0200 (CEST)
Subject: [erlang-bugs] ssh_cm:open_pty bug
In-Reply-To: <5e448700807101929s77a4369ax65d7bffd6621e3ef@mail.gmail.com>
References: <5e448700807101929s77a4369ax65d7bffd6621e3ef@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0807111039450.5675@mallor.du.uab.ericsson.se>


Hello!

Thank you for reporting this! We'll look in to this as soon as possible.

Best regards,

Niclas, Erlang/OTP

On Fri, 11 Jul 2008, Mikage Sawatari wrote:

> Hello,
>
> ssh_cm:open_pty has a bug.
> The attached patch corrects this problem.
>
> -- 
> -----------------------------------------------------------------------
> SAWATARI Mikage (SANO Taku)
>



From schlager@REDACTED  Fri Jul 11 14:44:09 2008
From: schlager@REDACTED (Tobias Schlager)
Date: Fri, 11 Jul 2008 14:44:09 +0200
Subject: [erlang-bugs] erl_interface: erl_match() doesn't match
	ERL_U_INTEGER ETERMs
Message-ID: <48775599.1020903@disy.net>

Hello,

I recently wondered why I'm getting incorrect return values when 
matching the same ETERM created with erl_mk_uint(). After looking in the 
OTP R12B-3 source code I found that ematch() in 
lib/erl_interface/src/legacy/erl_format.c is missing the case-clause for 
ERL_U_INTEGERs which leads to a result of false every time you try to 
match something with an unsigned integer ETERM as first argument.

I don't know if this really is a bug because matching two such big 
numbers may not be used very frequently but since it is possible to get 
unsigned integer values from an erlang node (imagine sending 
term_to_binary(16#EFFFFFFF) with a binary port_command), I would assume 
a correct behaviour just for convenience :)

Regards
Tobi

-- 

disy Informationssysteme GmbH
Dipl. Inform. Tobias Schlager
Erbprinzenstr. 4-12, 76133 Karlsruhe, Deutschland
Tel: +49 721 1 6006 607, E-Mail: schlager@REDACTED

Firmensitz: Erbprinzenstr. 4-12, 76133 Karlsruhe
Registergericht: Amtsgericht Mannheim, HRB 107964
Gesch?ftsf?hrer: Sven Behrens, Dr. Ralf Nikolai, Claus Hofmann

http://www.disy.net/cadenza - Berichte und Auswertungen mit Raumbezug
http://www.disy.net/e-meetings - Telefon- und Web-Konferenzen

--------------------------------
dialog . daten . dimensionen
--------------------------------



From JWatson@REDACTED  Fri Jul 11 17:35:49 2008
From: JWatson@REDACTED (John Watson)
Date: Fri, 11 Jul 2008 16:35:49 +0100
Subject: [erlang-bugs] FW: Scalar expressions problem in xmerl - xpath:scan
Message-ID: <85FA03E6BB6F0340B5A117013571A9B7FD7857@exchange3.ad.sis.tv>

Dear Erlang bug support,

 

I seem to be having problems using scalar expressions that include
functions within XPath in xmerl (1.1.9).  I am using R12B-3 on Windows.
The following XPath queries are legal, as far as I can tell, but xmerl
doesn't accept them. I'd be grateful if you could tell me if this really
is a problem and provide a fix if possible.  

 

Problem is in: 

 

xmerl_xpath:string(Pattern,Xml)

 

where Xml is a parsed reprsentation of Xml = "<code cd1= 'AA' cd2 = 'BB'
/>"

 

when Pattern is concat(/code/@cd1, /code/@cd2) it produces:

 

** exception error: no function clause matching

                    xmerl_xpath:match_expr({function_call,concat,

                                            [{path,abs,

                                              {refine,

                                               {step,

                                                {child,

 
{name,{code,[],"code"}},

                                                 []}},

                                               {step,

                                                {attribute,

                                                 {name,{cd1,[],"cd1"}},

                                                 []}}}},

                                             {path,abs,

                                              {refine,

                                               {step,

                                                {child,

 
{name,{code,[],"code"}},

                                                 []}},

                                               {step,

                                                {attribute,

                                                 {name,{cd2,[],"cd2"}},

                                                 []}}}}]},

                                           {state,

                                            {xmlContext,forward,

                                             {xmlNode,element,

                                              {xmlElement,code,code,[],

                                               {xmlNamespace,[],[]},

                                               [],1,

 
[{xmlAttribute,cd1,[],[],[],

                                                 [],1,[],"AA",false},

 
{xmlAttribute,cd2,[],[],[],

                                                 [],2,[],

                                                 [...],...}],

 
[],[],"C:/mycode/erlang/xml",

                                               undeclared},

                                              [],1},

                                             1,[],[],[],[],

                                             {xmlNode,root_node,

                                              {xmlDocument,

                                               {xmlElement,code,code,[],

                                                {xmlNamespace,[],[]},

                                                [],1,

                                                [{xmlAttribute,cd1,...},

                                                 {xmlAttribute,...}],

                                                [],[],

                                                [...],...}},

                                              [],1}},

                                            []})

     in function  xmerl_xpath:string/5

 

 

When Pattern is string-length(/code/@cd1) it produces:

 

** exception error: no function clause matching

 
xmerl_xpath:match_expr({function_call,'string-length',

                                            [{path,abs,

                                              {refine,

                                               {step,

                                                {child.......

 

     in function  xmerl_xpath:string/5

 

Yours sincerely

 

John Watson


*************************************************************
Satellite Information Services Limited Registered Office: 
17 Corsham Street London N1 6DR, Company No. 4243307

The information in this e-mail (which includes any files
transmitted with it) is confidential and is intended for the
addressee only. Unauthorised recipients are required to
maintain confidentiality.  If you have received this e-mail
in error please notify the sender immediately, destroy any
copies and delete it from your computer system. 
*************************************************************

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080711/0d839b53/attachment.htm>

From opfer@REDACTED  Sat Jul 12 11:13:54 2008
From: opfer@REDACTED (Christian Faulhammer)
Date: Sat, 12 Jul 2008 11:13:54 +0200
Subject: [erlang-bugs] crypto module incorrectly linked with DYNAMIC_SSL
Message-ID: <20080712111354.2555c57a@gentoo.org>

Hi,

a user of Gentoo Linux reported the following bug via our bug tracker:
<URL:http://bugs.gentoo.org/show_bug.cgi?id=231468>.  Summary is:

Erlang (BEAM) emulator version 5.6.3 [source] [smp:4] [async-threads:0]

Eshell V5.6.3  (abort with ^G)
1> crypto:start().
sh: line 0: exec: crypto_drv: not found
                                      
{error,{shutdown,{crypto_app,start,[normal,[]]}}}

We set SSL_DYNAMIC to yes duing build to have the system's libraries
used for linking.  Unfortunately this does not work.  

$ ldd /usr/lib/erlang/lib/crypto-1.5.2/priv/lib/crypto_drv.so
        linux-gate.so.1 =>  (0x5376b000)
        libc.so.6 => /lib/libc.so.6 (0x535b4000)
        /lib/ld-linux.so.2 (0x5376c000)

For his config.log file see the mentioned bug report.  Thanks for your
support.

V-Li

-- 
Christian Faulhammer, Gentoo Lisp project
<URL:http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode

<URL:http://www.faulhammer.org/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080712/e76f69bb/attachment.bin>

From s.egner@REDACTED  Mon Jul 14 11:35:56 2008
From: s.egner@REDACTED (Sebastian Egner)
Date: Mon, 14 Jul 2008 11:35:56 +0200
Subject: [erlang-bugs] Erlang on Windows NT4
Message-ID: <487B1DFC.5050001@specs.de>

Hello,

As the release notes for Erts 5.6.1 report under Section 1.4.1 
"Improvements and New Features",
Windows NT4 is not supported any longer by Erlang/OTP, i.e. from R12B incl.

Indeed, trying to install Erlang/OTP R12B-3 on a Windows-NT4 system reports:

    "The procedure entry point GetLongPathNameW could not be located in 
the dynamic link library KERNEL32.dll",

without any indication that this Windows OS is not supported, and no 
Erlang VM can be run.

Is this a bug, or does the NSIS installer simply not test for the 
version of the operating system?


More important, however, is my related question:

     *Are there any plans to support Windows NT4 again in the future?*

Let me briefly explain the background of this question: In November 2007 
our company (SPECS GmbH)
decided to invest in Erlang as a future platform for controlling our 
products, which are serious machines (up
to 2.5 M$) for studying the physical properties of solid surfaces. Our 
current software is a huge thing in Visual
C++ 6.0, glued together with CORBA and the Windows Registry. So far, our 
findings of using Erlang/OTP are
extremely encouraging. We have been able to interface with other parts 
of the software using CORBA and
Erlinterface with very little effort, and could write a layered system 
of drivers for ion guns, x-ray guns, etc. in
a very convenient and efficient fashion. (For the GUI we use Java/SWT.)

Unfortunately, our customers usually retire their old desktop PCs to 
their laboratories, and expect our
software to run on these platforms. In addition, we have lots of 
customers in countries where people
cannot afford to follow Microsoft's pace of product introduction, and 
generally do not upgrade their
PCs every year. Using a non-Microsoft OS is also not an option, because 
our customers use other
software with severe Microsoft lock-in for interpreting their 
experimental data.

Now if R11B-5 will be the last version to support Windows NT4, we have a 
very serious problem with
Erlang/OTP---in particular, because two significant bugs ('ic' 
generating unusable code and Erlinterface
not being able to communicate 0.0 correctly) that I had reported in 
R11B-5 are only fixed in R12B,
which came out 5-Dec-2007 and doesn't run on NT4 anymore (as I was 
surprised to learn today).

What advice would you give me on this situation?

Sebastian

P.S. Excuse me if you had this discussion already, and are tired of it 
being iterated at such a late point in time.

-- 

Dr. Sebastian Egner
Head of Software & 
Electronics Development

SPECS GmbH
Voltastr. 5
13355 Berlin
Germany

Phone: +49 (0)30 46 78 24 - 9210
Fax: +49 (0)30 46 42 083
E-Mail: s.egner@REDACTED   

Please visit www.specs.de

SPECS - YOUR COMPETENT PARTNER IN SURFACE ANALYSIS
----------------------------------------------------
SPECS Gesellschaft f?r Oberfl?chenanalytik und 
Computertechnologie mbH, CEO: Reinhard Lembke
Registered Office: Berlin
Companies Commercial Register No.: 93 HRB 21390 
Register Court: Charlottenburg
-----------------------------------------------------
This e-mail contains confidential and/or privileged
information. If you are not the intended recipient
(or have received this e-mail in error) please notify
the sender immediately and delete this e-mail. Any 
unauthorized copying, disclosure or distribution of
the material in this e-mail is strictly forbidden.



From toby@REDACTED  Mon Jul 14 19:22:03 2008
From: toby@REDACTED (Toby Thain)
Date: Mon, 14 Jul 2008 14:22:03 -0300
Subject: [erlang-bugs] Erlang on Windows NT4
In-Reply-To: <487B1DFC.5050001@specs.de>
References: <487B1DFC.5050001@specs.de>
Message-ID: <6BC376B6-590B-40AB-B08E-055E389E61AF@telegraphics.com.au>


On 14-Jul-08, at 6:35 AM, Sebastian Egner wrote:

> ...
>      *Are there any plans to support Windows NT4 again in the future?*
>
> Let me briefly explain the background of this question: ...
> software with severe Microsoft lock-in for interpreting their
> experimental data.

You certainly have my sympathy.

>
> Now if R11B-5 will be the last version to support Windows NT4, we  
> have a
> very serious problem with
> Erlang/OTP---in particular, because two significant bugs ('ic'
> generating unusable code and Erlinterface
> not being able to communicate 0.0 correctly) that I had reported in
> R11B-5 are only fixed in R12B,
> which came out 5-Dec-2007 and doesn't run on NT4 anymore (as I was
> surprised to learn today).
>
> What advice would you give me on this situation?

Paid support should achieve what you need?

--Toby

>
> Sebastian
>


From doug.mansell@REDACTED  Tue Jul 15 23:13:08 2008
From: doug.mansell@REDACTED (doug mansell)
Date: Tue, 15 Jul 2008 23:13:08 +0200
Subject: [erlang-bugs] Erlang on Windows NT4
In-Reply-To: <487B1DFC.5050001@specs.de>
References: <487B1DFC.5050001@specs.de>
Message-ID: <b721874c0807151413s1a12ac7atfb419985773ede4e@mail.gmail.com>

Hello,

Related to Sebastian's observations;  the docs on embedded systems
need updating too:

http://www.erlang.org/doc/embedded/embedded_nt.html#2

is still referring to NT 4 sp4 as the minimum Windows platform.

doug


On Mon, Jul 14, 2008 at 11:35 AM, Sebastian Egner <s.egner@REDACTED> wrote:
> Hello,
>
> As the release notes for Erts 5.6.1 report under Section 1.4.1
> "Improvements and New Features",
> Windows NT4 is not supported any longer by Erlang/OTP, i.e. from R12B incl.
>
> Indeed, trying to install Erlang/OTP R12B-3 on a Windows-NT4 system reports:
>
>    "The procedure entry point GetLongPathNameW could not be located in
> the dynamic link library KERNEL32.dll",
>
> without any indication that this Windows OS is not supported, and no
> Erlang VM can be run.


From paul-trapexit@REDACTED  Thu Jul 17 08:20:21 2008
From: paul-trapexit@REDACTED (Paul Mineiro)
Date: Wed, 16 Jul 2008 23:20:21 -0700 (PDT)
Subject: [erlang-bugs] erl_driver documentation confusing
Message-ID: <Pine.LNX.4.52L2.0807162304310.14460@herbie>

the documentation for erl_driver says that to ensure that the same thread
is used for all calls to your driver you should do:

r = driver_async(myPort, (unsigned int*)&myPort, myData, myFunc);

well this caused me lots of troubles because inside the implementation to
driver_async in erts/emulator/beam/erl_async.c there is a line

*key = qix;

where key is the 2nd argument to driver_async.  this caused some
segmentation faults in my code until i realized this was happening (since
I continued to use myPort as a valid port).

-- p

p.z. is there a reason *key is assigned at all?


From sverker@REDACTED  Thu Jul 17 14:51:05 2008
From: sverker@REDACTED (Sverker Eriksson)
Date: Thu, 17 Jul 2008 14:51:05 +0200
Subject: [erlang-bugs] erl_driver documentation confusing
In-Reply-To: <Pine.LNX.4.52L2.0807162304310.14460@herbie>
References: <Pine.LNX.4.52L2.0807162304310.14460@herbie>
Message-ID: <487F4039.6080100@erix.ericsson.se>

Hi Paul

I agree the example is a bit misleading. I guess this is what you want 
to do:

unsigned key = (unsigned) myPort;

:

r = driver_async(myPort, &key, myData, myFunc);


Using the port identifier is just a way to select a "random" thread. And 
it's enough to initialize 'key' once, in the start function of 
erl_drv_entry for example.

The only reason for driver_async to assign to *key (that I can see) is 
that, after the call to driver_async(), *key will uniquely identify the 
used thread. I don't see how that would be very useful though.

I will change the documentation according to my example above, unless 
you have any other suggestion.

/Sverker, Erlang/OTP Ericsson


Paul Mineiro wrote:
> the documentation for erl_driver says that to ensure that the same thread
> is used for all calls to your driver you should do:
>
> r = driver_async(myPort, (unsigned int*)&myPort, myData, myFunc);
>
> well this caused me lots of troubles because inside the implementation to
> driver_async in erts/emulator/beam/erl_async.c there is a line
>
> *key = qix;
>
> where key is the 2nd argument to driver_async.  this caused some
> segmentation faults in my code until i realized this was happening (since
> I continued to use myPort as a valid port).
>
> -- p
>
> p.z. is there a reason *key is assigned at all?
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
>   



From s.egner@REDACTED  Thu Jul 17 15:21:32 2008
From: s.egner@REDACTED (Sebastian Egner)
Date: Thu, 17 Jul 2008 15:21:32 +0200
Subject: [erlang-bugs] Erlang on Windows NT4
Message-ID: <487F475C.2050506@specs.de>

Hello,

Thank you for all the comments!

For what it's worth, I found out that the NT4 compatibility issue is not 
related to Erlang/OTP specifically, but rather an general byproduct of 
Microsoft terminating support for NT4 in Visual C++ 8:

    
http://forums.msdn.microsoft.com/en-US/vcgeneral/thread/2435c3ab-f732-467e-8224-5f4e3f12c10b/

    
http://www.mombu.com/microsoft/windows-programmer-win32/t-vs2005-and-nt4-392831.html

In short, when the system loads the VC8 runtime lib MSVCR80.DLL, it 
tries to resolve a reference to the function GetLongPathNameW in 
KERNEL32.DLL, which NT4 doesn't have. The beauty of it is that this 
reference seems to be the only reason for most VC8 application not to 
run on NT4! People have reported successfully running non-unicode  VC8 
applications by using custom-compiled versions of MSVCR80.DLL without 
the call to GetLongPathNameW.

This means, short of ERTS being ported back to VC 6.0, there is no hope 
of an 'official' version of ERTS supporting NT4, because Microsoft's 
licence will prevent Ericsson to include a patched runtime lib in the 
redistributable VC package. (Unless I am mistaken and there is a legal 
way of supporting NT with VC8.)

Experimentally trying out R12B-3 with a patched version of MSVCR80.DLL 
(<dirty mode="don't try this at home">Start emacs, load MSVCR80.DLL, 
replace "GetLongPathNameW" by "GetModuleHandleA" (= 16 chars and 
linkable), save.</dirty>), and creating a proper 'erl.ini' (which the 
installer could not do due to an acute lack of working VMs at the time), 
I am informed by the VM:

    C:\Program Files\erl5.6.3\erts-5.6.3\bin>erl
    Windows version not supported (min required: winnt 5.0)

So this clarifies the bug-report of my first email:

Yes, Erlang/OTP R12B-3 does check the OS version and refuses to run on 
NT4. However, the check is executed in an Erlang VM, which won't start 
in the first place, throwing an error message that takes an expert to 
interpret.

You could consider checking the OS version in the NSIS installer itself, 
e.g. using the function GetWindowsVersion:

    http://nsis.sourceforge.net/Get_Windows_version

and deactivate the check from the VM (erts\emulator\sys\win32\sys.c: 
check_supported_os_version()).

Kind regards,

Sebastian


From erlang-questions_efine@REDACTED  Thu Jul 17 17:57:43 2008
From: erlang-questions_efine@REDACTED (Edwin Fine)
Date: Thu, 17 Jul 2008 11:57:43 -0400
Subject: [erlang-bugs] Erlang on Windows NT4
In-Reply-To: <487F475C.2050506@specs.de>
References: <487F475C.2050506@specs.de>
Message-ID: <6c2563b20807170857n17411f07o594fc31bf7c7b75f@mail.gmail.com>

Couldn't you create a new "compatibility" DLL, that implements or stubs
"GetLongPathNameW"? Then if you link in that DLL to the Erlang NT4 build,
the call will resolve at run time. This is much less dirty, and if Erlang
adds it in as an NT4 compatibility patch, everyone will be happy.

On Thu, Jul 17, 2008 at 9:21 AM, Sebastian Egner <s.egner@REDACTED> wrote:

> Hello,
>
> Thank you for all the comments!
>
> For what it's worth, I found out that the NT4 compatibility issue is not
> related to Erlang/OTP specifically, but rather an general byproduct of
> Microsoft terminating support for NT4 in Visual C++ 8:
>
>
>
> http://forums.msdn.microsoft.com/en-US/vcgeneral/thread/2435c3ab-f732-467e-8224-5f4e3f12c10b/
>
>
>
> http://www.mombu.com/microsoft/windows-programmer-win32/t-vs2005-and-nt4-392831.html
>
> In short, when the system loads the VC8 runtime lib MSVCR80.DLL, it
> tries to resolve a reference to the function GetLongPathNameW in
> KERNEL32.DLL, which NT4 doesn't have. The beauty of it is that this
> reference seems to be the only reason for most VC8 application not to
> run on NT4! People have reported successfully running non-unicode  VC8
> applications by using custom-compiled versions of MSVCR80.DLL without
> the call to GetLongPathNameW.
>
> This means, short of ERTS being ported back to VC 6.0, there is no hope
> of an 'official' version of ERTS supporting NT4, because Microsoft's
> licence will prevent Ericsson to include a patched runtime lib in the
> redistributable VC package. (Unless I am mistaken and there is a legal
> way of supporting NT with VC8.)
>
> Experimentally trying out R12B-3 with a patched version of MSVCR80.DLL
> (<dirty mode="don't try this at home">Start emacs, load MSVCR80.DLL,
> replace "GetLongPathNameW" by "GetModuleHandleA" (= 16 chars and
> linkable), save.</dirty>), and creating a proper 'erl.ini' (which the
> installer could not do due to an acute lack of working VMs at the time),
> I am informed by the VM:
>
>    C:\Program Files\erl5.6.3\erts-5.6.3\bin>erl
>    Windows version not supported (min required: winnt 5.0)
>
> So this clarifies the bug-report of my first email:
>
> Yes, Erlang/OTP R12B-3 does check the OS version and refuses to run on
> NT4. However, the check is executed in an Erlang VM, which won't start
> in the first place, throwing an error message that takes an expert to
> interpret.
>
> You could consider checking the OS version in the NSIS installer itself,
> e.g. using the function GetWindowsVersion:
>
>    http://nsis.sourceforge.net/Get_Windows_version
>
> and deactivate the check from the VM (erts\emulator\sys\win32\sys.c:
> check_supported_os_version()).
>
> Kind regards,
>
> Sebastian
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
>
>


-- 
The great enemy of the truth is very often not the lie -- deliberate,
contrived and dishonest, but the myth, persistent, persuasive, and
unrealistic. Belief in myths allows the comfort of opinion without the
discomfort of thought.
John F. Kennedy 35th president of US 1961-1963 (1917 - 1963)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080717/1ebba4b8/attachment.htm>

From paul-trapexit@REDACTED  Thu Jul 17 19:04:54 2008
From: paul-trapexit@REDACTED (Paul Mineiro)
Date: Thu, 17 Jul 2008 10:04:54 -0700 (PDT)
Subject: [erlang-bugs] erl_driver documentation confusing
In-Reply-To: <487F4039.6080100@erix.ericsson.se>
References: <Pine.LNX.4.52L2.0807162304310.14460@herbie>
	<487F4039.6080100@erix.ericsson.se>
Message-ID: <Pine.LNX.4.52L2.0807170958370.14460@herbie>

Yes, that's essentially what I ended up doing.  That documentation change
will save the next person some time, I imagine :)

Re: other suggestions, it's too dangerous to propose actually changing the
behaviour, maybe somebody in the bowels of some linked-in driver is
counting on it.  So I'm fine with just the documentation change.

-- p

On Thu, 17 Jul 2008, Sverker Eriksson wrote:

> Hi Paul
>
> I agree the example is a bit misleading. I guess this is what you want
> to do:
>
> unsigned key = (unsigned) myPort;
>
> :
>
> r = driver_async(myPort, &key, myData, myFunc);
>
>
> Using the port identifier is just a way to select a "random" thread. And
> it's enough to initialize 'key' once, in the start function of
> erl_drv_entry for example.
>
> The only reason for driver_async to assign to *key (that I can see) is
> that, after the call to driver_async(), *key will uniquely identify the
> used thread. I don't see how that would be very useful though.
>
> I will change the documentation according to my example above, unless
> you have any other suggestion.
>
> /Sverker, Erlang/OTP Ericsson
>
>
> Paul Mineiro wrote:
> > the documentation for erl_driver says that to ensure that the same thread
> > is used for all calls to your driver you should do:
> >
> > r = driver_async(myPort, (unsigned int*)&myPort, myData, myFunc);
> >
> > well this caused me lots of troubles because inside the implementation to
> > driver_async in erts/emulator/beam/erl_async.c there is a line
> >
> > *key = qix;
> >
> > where key is the 2nd argument to driver_async.  this caused some
> > segmentation faults in my code until i realized this was happening (since
> > I continued to use myPort as a valid port).
> >
> > -- p
> >
> > p.z. is there a reason *key is assigned at all?
> > _______________________________________________
> > erlang-bugs mailing list
> > erlang-bugs@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-bugs
> >
>
>

In an artificial world, only extremists live naturally.

        -- Paul Graham


From sgolovan@REDACTED  Sun Jul 20 06:28:46 2008
From: sgolovan@REDACTED (Sergei Golovan)
Date: Sun, 20 Jul 2008 08:28:46 +0400
Subject: [erlang-bugs] Incomplete detaching from a controlling terminal
Message-ID: <f60b7eb60807192128v6283850dne3824d6ea841492f@mail.gmail.com>

Hi!

Appears that if started with -detached option erlexec doesn't fully
detach from a controlling terminal which may end with killing a
detached service. For example, in Debian GNU/Linux (current unstable
or testing) installing an Erlang-based service (e.g. YAWS) leads to
killing the yaws daemon just after apt-get finishes.

I've asked in debian-devel mailing list showing the following code from erlexec:

 if (start_detached) {
   int status = fork();
   if (status != 0)
     return 0;
   status = fork();
   if (status != 0)
     return 0;

   close(0);
   open("/dev/null", O_RDONLY);
   close(1);
   open("/dev/null", O_WRONLY);
   close(2);
   open("/dev/null", O_WRONLY);
 }
 {
   execv(emu, Eargsp); /* executing the main Erlang emulator */
 }

and the answer was that a call to setsid() is missing just before the
seconf fork(), so the session leader remains unchanged and if it
finishes the current daemonized process gets a signal.

The minimal patch which fixes the issue for me is the following:

--- erts/etc/common/erlexec.c.orig
+++ erts/etc/common/erlexec.c
@@ -913,6 +913,7 @@
 	int status = fork();
 	if (status != 0)	/* Parent */
 	    return 0;
+	setsid();
 	status = fork();
 	if (status != 0)	/* Parent */
 	    return 0;

Though there could be the case when setsid() isn't available and more
sophisticated technique is needed. Russ Allbery in
http://lists.debian.org/debian-devel/2008/07/msg00623.html proposed a
function daemon() which is to be accompanied by checks for setsid()
and TIOCNOTTY ioctl.

Cheers!
-- 
Sergei Golovan


From eric.melbardis@REDACTED  Sun Jul 20 14:05:10 2008
From: eric.melbardis@REDACTED (Eric P. Melbardis)
Date: Sun, 20 Jul 2008 05:05:10 -0700
Subject: [erlang-bugs] r12b werl behaviour after application termination
	event
Message-ID: <003701c8ea60$dc54e3d0$94feab70$@melbardis@netkitsolutions.com>

Hi

 

I have just upgraded from r10b-10 to r12b-3.

 

I just noticed that if I get an application termination failure in my
application, then closing the werl window will not cause werl to actually
exit. I still have to go to the task manager and kill the process manually.
If I simply start werl plainly, and have no error, then closing the window
will terminate the process.

 

I am running windows xp/sp3 (32-bit) erlang compiled for smp, and I have 2
cores.

 

This behavior is different from before, is it a bug??

 

Regards

 

Eric

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080720/46740316/attachment.htm>

From taisuke.hori+erlang-bugs@REDACTED  Sun Jul 20 20:02:31 2008
From: taisuke.hori+erlang-bugs@REDACTED (Taisuke Hori)
Date: Mon, 21 Jul 2008 03:02:31 +0900
Subject: [erlang-bugs] regexp: no pattern matches empty string
In-Reply-To: <36278dc50807152240s75ebabd4tc7e23d7c5f2f63ea@mail.gmail.com>
References: <36278dc50807152240s75ebabd4tc7e23d7c5f2f63ea@mail.gmail.com>
Message-ID: <36278dc50807201102o32fb2481r4350b02b078cf1ea@mail.gmail.com>

Hello,

I suspect a bug in regexp. I tried to match some patterns to a empty
list, but they ware not matched.

$ erl -version
Erlang (ASYNC_THREADS,HIPE) (BEAM) emulator version 5.5.5
$ erl
Eshell V5.5.5  (abort with ^G)
1> regexp:match("", "").
nomatch
2> regexp:match("", ".*").
nomatch

Is this a bug or feature?

Thank you


--
Taisuke HORI
mail: taisuke.hori@REDACTED



-- 
Taisuke Hori
taisuke.hori@REDACTED


From paul-trapexit@REDACTED  Sun Jul 20 23:23:17 2008
From: paul-trapexit@REDACTED (Paul Mineiro)
Date: Sun, 20 Jul 2008 14:23:17 -0700 (PDT)
Subject: [erlang-bugs] mnesia_loader:db_match_erase/2 can only badmatch with
	dets
Message-ID: <Pine.LNX.4.52L2.0807201419190.14460@herbie>

The return value of dets:match_delete/2 is an integer, or an error tuple;
mnesia_loader is matching against ok.

Attached is a suggested patch.

-- p

p.z. I wonder if the dialyzer would have found this ...
-------------- next part --------------
--- /Users/pmineiro/src/otp_src_R12B-2/lib/mnesia/src/mnesia_loader.erl.orig	2008-07-20 14:17:25.000000000 -0700
+++ mnesia_loader.erl	2008-07-20 14:20:24.000000000 -0700
@@ -620,7 +620,9 @@
 db_match_erase({disc_copies, Tab} , Pat) ->
     true = ?ets_match_delete(Tab, Pat);
 db_match_erase({disc_only_copies, Tab}, Pat) ->
-    ok = dets:match_delete(Tab, Pat).
+    case dets:match_delete(Tab, Pat) of
+      N when is_integer (N) -> true
+    end.
 
 db_put({ram_copies, Tab}, Val) ->
     true = ?ets_insert(Tab, Val);

From paul-trapexit@REDACTED  Sun Jul 20 23:30:45 2008
From: paul-trapexit@REDACTED (Paul Mineiro)
Date: Sun, 20 Jul 2008 14:30:45 -0700 (PDT)
Subject: [erlang-bugs] mnesia_loader:db_match_erase/2 can only badmatch
	with dets
In-Reply-To: <Pine.LNX.4.52L2.0807201419190.14460@herbie>
References: <Pine.LNX.4.52L2.0807201419190.14460@herbie>
Message-ID: <Pine.LNX.4.52L2.0807201429560.14460@herbie>

Heads up, I found another instance of the same problem in
mnesia_index:db_match_erase/2 .

-- p

On Sun, 20 Jul 2008, Paul Mineiro wrote:

> The return value of dets:match_delete/2 is an integer, or an error tuple;
> mnesia_loader is matching against ok.
>
> Attached is a suggested patch.
>
> -- p
>
> p.z. I wonder if the dialyzer would have found this ...

In an artificial world, only extremists live naturally.

        -- Paul Graham


From kenji.rikitake@REDACTED  Mon Jul 21 09:13:08 2008
From: kenji.rikitake@REDACTED (Kenji Rikitake)
Date: Mon, 21 Jul 2008 16:13:08 +0900
Subject: [erlang-bugs] R12B3 epmd debug message typo
Message-ID: <20080721071308.GA39531@k2r.org>

I've found in R12B3 the word "epmd" are typed in as "empd".
This is confusing and should be fixed.
A patch included in this message.

Kenji Rikitake

-------------- next part --------------
A non-text attachment was scrubbed...
Name: epmd_srv.c.typobug.diff
Type: text/x-diff
Size: 806 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080721/5a97990e/attachment.bin>

From nick@REDACTED  Mon Jul 21 10:59:41 2008
From: nick@REDACTED (Niclas Eklund)
Date: Mon, 21 Jul 2008 10:59:41 +0200 (CEST)
Subject: [erlang-bugs] mnesia_loader:db_match_erase/2 can only badmatch
 with dets
In-Reply-To: <Pine.LNX.4.52L2.0807201429560.14460@herbie>
References: <Pine.LNX.4.52L2.0807201419190.14460@herbie>
	<Pine.LNX.4.52L2.0807201429560.14460@herbie>
Message-ID: <Pine.LNX.4.64.0807211040510.5675@mallor.du.uab.ericsson.se>


Hello!

Thank you for reporting this! This might simply be a documentation bug 
since dets:match_delete/2 can return ok (see below). I'll notify the 
person responsible for the Dets application about this.

Best Regards,

Niclas Erlang/OTP


%% dets.erl
match_delete(Tab, Pat) ->
     badarg(match_delete(Tab, Pat, delete), [Tab, Pat]).

match_delete(Tab, Pat, What) ->
     safe_fixtable(Tab, true),
     case compile_match_spec(What, Pat) of
 	{Spec, MP} ->
 	    Proc = dets_server:get_pid(Tab),
 	    R = req(Proc, {match_delete_init, MP, Spec}),
 	    do_match_delete(Tab, Proc, R, What, 0);
 	badarg ->
 	    badarg
     end.

do_match_delete(Tab, _Proc, {done, N1}, select, N) ->
     safe_fixtable(Tab, false),
     N + N1;
do_match_delete(Tab, _Proc, {done, _N1}, _What, _N) ->
     safe_fixtable(Tab, false),
     ok;
do_match_delete(Tab, Proc, {cont, State, N1}, What, N) ->
     do_match_delete(Tab, Proc, req(Proc, {match_delete, State}), What, 
N+N1);
do_match_delete(Tab, _Proc, Error, _What, _N) ->
     safe_fixtable(Tab, false),
     Error.



On Sun, 20 Jul 2008, Paul Mineiro wrote:

> Heads up, I found another instance of the same problem in
> mnesia_index:db_match_erase/2 .
>
> -- p
>
> On Sun, 20 Jul 2008, Paul Mineiro wrote:
>
>> The return value of dets:match_delete/2 is an integer, or an error tuple;
>> mnesia_loader is matching against ok.
>>
>> Attached is a suggested patch.
>>
>> -- p
>>
>> p.z. I wonder if the dialyzer would have found this ...
>
> In an artificial world, only extremists live naturally.
>
>        -- Paul Graham
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
>



From jlist@REDACTED  Wed Jul 23 01:02:13 2008
From: jlist@REDACTED (jlist@REDACTED)
Date: Tue, 22 Jul 2008 16:02:13 -0700
Subject: [erlang-bugs] Segfault in erl_interface attempting to decode
	certain large binaries
Message-ID: <065901c8ec4e$fa744ff0$ef5cefd0$@com>

All,

I have a TCP interface between an Erlang system and a C system.? Both
send/receive marshaled binary Erlang terms and I have not had any problems
to date.

Today I began doing some more serious testing with larger chunks of binary
to be decoded in C.

We ran into a bug (it seems) with erl_interface 3.5.5.4 that is causing it
to segfault during decoding.? The backtrace looks like this:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 46912496233216 (LWP 4091)]
0x00000000004032c4 in _erl_free_term ()
(gdb) bt
#0? 0x00000000004032c4 in _erl_free_term ()
#1? 0x000000000040496b in erl_decode_it ()
#2? 0x0000000000404937 in erl_decode_it ()
#3? 0x0000000000404c93 in erl_decode_it ()
#4? 0x0000000000404c93 in erl_decode_it ()
#5? 0x0000000000405311 in erl_decode ()
#6? 0x0000000000401b38 in main (argc=1, argv=0x7fff51ec7938) at badtest.c:28

The unfortunate part is that the way this large binary term is generated
cannot be done in any kind of sample code (it?s being pulled off an external
database).

Testing code:  http://jgray.la/erlang/erl_decode_segfault_test.tar.gz

However, I have created a set of test files in C which recreate the
segfault.? I stored the binary in a flat file (as ?badbinary?) and have a
testing program which reads it off disk and attempts to decode it.? To prove
the approach is sane (and that this segfault is related to something strange
about the decoding of this particular binary, not the size or general format
of the binary) there is a ?goodbinary? file and testing program for that.

To use the test code:

Untar/Ungzip the file.? You may need to edit the Makefile to fix the paths
to your erl_interface library.

?make? and then you can:

./badtest? (this reads ?badbinary? and attempts to decode, causes segfault)
./goodtest (this reads ?goodbinary? and successfully decodes it)? [nearly
identical code to badtest.c but reads different file w/ different size]

Also included is

./makegoodbin (a simple program that generates a large ETERM in an identical
format to the badbinary but contains duplicated binary data everywhere) 


Notes:

* The marshaled binary erlang term being sent to C can be successfully
decoded/unmarshaled from within Erlang without a problem
* This is reproducible with many different large erlang terms generated from
our database queries. ??makegoodbin.c? creates a term identical in format to
those causing problems, however it does not have the random distribution of
binary sizes and content, and so I?m not able to reproduce the problem in
this way.
* The entire system, end-to-end including this decoding step, works
perfectly in most cases.? However when the data goes into the 100k+ range,
the segfaults start to happen.? That?s why I created the ?makegoodbin? which
follows the same format.? Unfortunately that works even at sizes of >1MB
adding to the confusion of the problem.


Any help is appreciated.? Thanks.

I apologize if this is a repost.  I never saw my original post hit the list
and did not receive any responses.

Jonathan Gray
Streamy Inc.



From saleyn@REDACTED  Wed Jul 23 12:51:40 2008
From: saleyn@REDACTED (Serge Aleynikov)
Date: Wed, 23 Jul 2008 06:51:40 -0400
Subject: [erlang-bugs] Segfault in erl_interface attempting to decode
 certain large binaries
In-Reply-To: <065901c8ec4e$fa744ff0$ef5cefd0$@com>
References: <065901c8ec4e$fa744ff0$ef5cefd0$@com>
Message-ID: <48870D3C.60606@gmail.com>

Jonathan,

It's not clear from your description if you're using erl_interface to 
build a driver or a port program.  While I don't have an answer to your 
direct question, perhaps if you are writing a C port program you can use 
ei instead of erl_interface, and if you are writing a driver, you can 
use driver_output_term() / driver_send_term() and corresponding 
ErlDrvTermData* structures to pass data to/from the emulator (*). This 
is the fastest way to communicate with the emulator's port owner process.

(*) See: http://www.erlang.org/doc/man/erl_driver.html
and also the source code of inet_drv.c in the distribution for various 
LOAD_*() macros that simplify working with ErlDrvTermData structures.

Serge




jlist@REDACTED wrote:
> All,
> 
> I have a TCP interface between an Erlang system and a C system.  Both
> send/receive marshaled binary Erlang terms and I have not had any problems
> to date.
> 
> Today I began doing some more serious testing with larger chunks of binary
> to be decoded in C.
> 
> We ran into a bug (it seems) with erl_interface 3.5.5.4 that is causing it
> to segfault during decoding.  The backtrace looks like this:
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 46912496233216 (LWP 4091)]
> 0x00000000004032c4 in _erl_free_term ()
> (gdb) bt
> #0  0x00000000004032c4 in _erl_free_term ()
> #1  0x000000000040496b in erl_decode_it ()
> #2  0x0000000000404937 in erl_decode_it ()
> #3  0x0000000000404c93 in erl_decode_it ()
> #4  0x0000000000404c93 in erl_decode_it ()
> #5  0x0000000000405311 in erl_decode ()
> #6  0x0000000000401b38 in main (argc=1, argv=0x7fff51ec7938) at badtest.c:28
> 
> The unfortunate part is that the way this large binary term is generated
> cannot be done in any kind of sample code (it?s being pulled off an external
> database).
> 
> Testing code:  http://jgray.la/erlang/erl_decode_segfault_test.tar.gz
> 
> However, I have created a set of test files in C which recreate the
> segfault.  I stored the binary in a flat file (as ?badbinary?) and have a
> testing program which reads it off disk and attempts to decode it.  To prove
> the approach is sane (and that this segfault is related to something strange
> about the decoding of this particular binary, not the size or general format
> of the binary) there is a ?goodbinary? file and testing program for that.
> 
> To use the test code:
> 
> Untar/Ungzip the file.  You may need to edit the Makefile to fix the paths
> to your erl_interface library.
> 
> ?make? and then you can:
> 
> ./badtest  (this reads ?badbinary? and attempts to decode, causes segfault)
> ./goodtest (this reads ?goodbinary? and successfully decodes it)  [nearly
> identical code to badtest.c but reads different file w/ different size]
> 
> Also included is
> 
> ./makegoodbin (a simple program that generates a large ETERM in an identical
> format to the badbinary but contains duplicated binary data everywhere) 
> 
> 
> Notes:
> 
> * The marshaled binary erlang term being sent to C can be successfully
> decoded/unmarshaled from within Erlang without a problem
> * This is reproducible with many different large erlang terms generated from
> our database queries.  ?makegoodbin.c? creates a term identical in format to
> those causing problems, however it does not have the random distribution of
> binary sizes and content, and so I?m not able to reproduce the problem in
> this way.
> * The entire system, end-to-end including this decoding step, works
> perfectly in most cases.  However when the data goes into the 100k+ range,
> the segfaults start to happen.  That?s why I created the ?makegoodbin? which
> follows the same format.  Unfortunately that works even at sizes of >1MB
> adding to the confusion of the problem.
> 
> 
> Any help is appreciated.  Thanks.
> 
> I apologize if this is a repost.  I never saw my original post hit the list
> and did not receive any responses.
> 
> Jonathan Gray
> Streamy Inc.
> 
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
> 



From jlist@REDACTED  Wed Jul 23 17:50:20 2008
From: jlist@REDACTED (Jonathan Gray)
Date: Wed, 23 Jul 2008 08:50:20 -0700
Subject: [erlang-bugs] Segfault in erl_interface attempting to decode
	certain large binaries
In-Reply-To: <48870D3C.60606@gmail.com>
References: <065901c8ec4e$fa744ff0$ef5cefd0$@com> <48870D3C.60606@gmail.com>
Message-ID: <06e801c8ecdb$cf94af40$6ebe0dc0$@com>

Serge,

Sorry I was not more clear.  This is a port program with two remote systems
communicating via simple TCP packets (size header, buffer).

Thanks for the advice.  The documentation suggests that erl_interface is
actually making use of the ei code.  I'm not sure if that's correct because
I see other resources on the web that say they are different (ei is new,
erl_interface is old).  However the error does exist in a malloc() call so
perhaps that can be avoided doing the term decodes myself?

Unfortunately what I need (already have working but segfaulting on a large
decode) is a generic parser that converts ErlBinary into a number of
different formats (JSON, Python, C struct tree) and vice versa.  The ETERM
representation gave me an easy way to make recursive converters.  Recreating
this generic behavior using ei seems like I'll be recreating functions of
erl_interface and losing the ETERM representation so I'll need to rewrite my
converters in a completely different way. 

Rewriting it to go directly from binary would be a good thing in the long
run, just not something I was planning on doing at this stage.  Has anyone
ever written a newer generic erl_encode/erl_decode using the latest ei?

I will take a closer look at ei and report back.

Thanks for your help.

Jonathan


-----Original Message-----
From: erlang-bugs-bounces@REDACTED [mailto:erlang-bugs-bounces@REDACTED]
On Behalf Of Serge Aleynikov
Sent: Wednesday, July 23, 2008 3:52 AM
To: jlist@REDACTED
Cc: erlang-bugs@REDACTED
Subject: Re: [erlang-bugs] Segfault in erl_interface attempting to decode
certain large binaries

Jonathan,

It's not clear from your description if you're using erl_interface to 
build a driver or a port program.  While I don't have an answer to your 
direct question, perhaps if you are writing a C port program you can use 
ei instead of erl_interface, and if you are writing a driver, you can 
use driver_output_term() / driver_send_term() and corresponding 
ErlDrvTermData* structures to pass data to/from the emulator (*). This 
is the fastest way to communicate with the emulator's port owner process.

(*) See: http://www.erlang.org/doc/man/erl_driver.html
and also the source code of inet_drv.c in the distribution for various 
LOAD_*() macros that simplify working with ErlDrvTermData structures.

Serge




jlist@REDACTED wrote:
> All,
> 
> I have a TCP interface between an Erlang system and a C system.  Both
> send/receive marshaled binary Erlang terms and I have not had any problems
> to date.
> 
> Today I began doing some more serious testing with larger chunks of binary
> to be decoded in C.
> 
> We ran into a bug (it seems) with erl_interface 3.5.5.4 that is causing it
> to segfault during decoding.  The backtrace looks like this:
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 46912496233216 (LWP 4091)]
> 0x00000000004032c4 in _erl_free_term ()
> (gdb) bt
> #0  0x00000000004032c4 in _erl_free_term ()
> #1  0x000000000040496b in erl_decode_it ()
> #2  0x0000000000404937 in erl_decode_it ()
> #3  0x0000000000404c93 in erl_decode_it ()
> #4  0x0000000000404c93 in erl_decode_it ()
> #5  0x0000000000405311 in erl_decode ()
> #6  0x0000000000401b38 in main (argc=1, argv=0x7fff51ec7938) at
badtest.c:28
> 
> The unfortunate part is that the way this large binary term is generated
> cannot be done in any kind of sample code (it's being pulled off an
external
> database).
> 
> Testing code:  http://jgray.la/erlang/erl_decode_segfault_test.tar.gz
> 
> However, I have created a set of test files in C which recreate the
> segfault.  I stored the binary in a flat file (as 'badbinary') and have a
> testing program which reads it off disk and attempts to decode it.  To
prove
> the approach is sane (and that this segfault is related to something
strange
> about the decoding of this particular binary, not the size or general
format
> of the binary) there is a 'goodbinary' file and testing program for that.
> 
> To use the test code:
> 
> Untar/Ungzip the file.  You may need to edit the Makefile to fix the paths
> to your erl_interface library.
> 
> 'make' and then you can:
> 
> ./badtest  (this reads 'badbinary' and attempts to decode, causes
segfault)
> ./goodtest (this reads 'goodbinary' and successfully decodes it)  [nearly
> identical code to badtest.c but reads different file w/ different size]
> 
> Also included is
> 
> ./makegoodbin (a simple program that generates a large ETERM in an
identical
> format to the badbinary but contains duplicated binary data everywhere) 
> 
> 
> Notes:
> 
> * The marshaled binary erlang term being sent to C can be successfully
> decoded/unmarshaled from within Erlang without a problem
> * This is reproducible with many different large erlang terms generated
from
> our database queries.  'makegoodbin.c' creates a term identical in format
to
> those causing problems, however it does not have the random distribution
of
> binary sizes and content, and so I'm not able to reproduce the problem in
> this way.
> * The entire system, end-to-end including this decoding step, works
> perfectly in most cases.  However when the data goes into the 100k+ range,
> the segfaults start to happen.  That's why I created the 'makegoodbin'
which
> follows the same format.  Unfortunately that works even at sizes of >1MB
> adding to the confusion of the problem.
> 
> 
> Any help is appreciated.  Thanks.
> 
> I apologize if this is a repost.  I never saw my original post hit the
list
> and did not receive any responses.
> 
> Jonathan Gray
> Streamy Inc.
> 
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
> 

_______________________________________________
erlang-bugs mailing list
erlang-bugs@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-bugs



From saleyn@REDACTED  Thu Jul 24 02:51:16 2008
From: saleyn@REDACTED (Serge Aleynikov)
Date: Wed, 23 Jul 2008 20:51:16 -0400
Subject: [erlang-bugs] Segfault in erl_interface attempting to decode
 certain large binaries
In-Reply-To: <06e801c8ecdb$cf94af40$6ebe0dc0$@com>
References: <065901c8ec4e$fa744ff0$ef5cefd0$@com> <48870D3C.60606@gmail.com>
	<06e801c8ecdb$cf94af40$6ebe0dc0$@com>
Message-ID: <4887D204.6020002@gmail.com>

You can use the following tutorial to help you started with ei:
http://www.trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_programs

I haven't seen any newer erl_encode/erl_decode implementations.  Frankly 
I've started doing that myself in C++ and got the recursive encoding 
part, but haven't done the decoder and had to put this project on hold 
due to other priorities.

Serge

Jonathan Gray wrote:
> Serge,
> 
> Sorry I was not more clear.  This is a port program with two remote systems
> communicating via simple TCP packets (size header, buffer).
> 
> Thanks for the advice.  The documentation suggests that erl_interface is
> actually making use of the ei code.  I'm not sure if that's correct because
> I see other resources on the web that say they are different (ei is new,
> erl_interface is old).  However the error does exist in a malloc() call so
> perhaps that can be avoided doing the term decodes myself?
> 
> Unfortunately what I need (already have working but segfaulting on a large
> decode) is a generic parser that converts ErlBinary into a number of
> different formats (JSON, Python, C struct tree) and vice versa.  The ETERM
> representation gave me an easy way to make recursive converters.  Recreating
> this generic behavior using ei seems like I'll be recreating functions of
> erl_interface and losing the ETERM representation so I'll need to rewrite my
> converters in a completely different way. 
> 
> Rewriting it to go directly from binary would be a good thing in the long
> run, just not something I was planning on doing at this stage.  Has anyone
> ever written a newer generic erl_encode/erl_decode using the latest ei?
> 
> I will take a closer look at ei and report back.
> 
> Thanks for your help.
> 
> Jonathan
> 
> 
> -----Original Message-----
> From: erlang-bugs-bounces@REDACTED [mailto:erlang-bugs-bounces@REDACTED]
> On Behalf Of Serge Aleynikov
> Sent: Wednesday, July 23, 2008 3:52 AM
> To: jlist@REDACTED
> Cc: erlang-bugs@REDACTED
> Subject: Re: [erlang-bugs] Segfault in erl_interface attempting to decode
> certain large binaries
> 
> Jonathan,
> 
> It's not clear from your description if you're using erl_interface to 
> build a driver or a port program.  While I don't have an answer to your 
> direct question, perhaps if you are writing a C port program you can use 
> ei instead of erl_interface, and if you are writing a driver, you can 
> use driver_output_term() / driver_send_term() and corresponding 
> ErlDrvTermData* structures to pass data to/from the emulator (*). This 
> is the fastest way to communicate with the emulator's port owner process.
> 
> (*) See: http://www.erlang.org/doc/man/erl_driver.html
> and also the source code of inet_drv.c in the distribution for various 
> LOAD_*() macros that simplify working with ErlDrvTermData structures.
> 
> Serge
> 
> 
> 
> 
> jlist@REDACTED wrote:
>> All,
>>
>> I have a TCP interface between an Erlang system and a C system.  Both
>> send/receive marshaled binary Erlang terms and I have not had any problems
>> to date.
>>
>> Today I began doing some more serious testing with larger chunks of binary
>> to be decoded in C.
>>
>> We ran into a bug (it seems) with erl_interface 3.5.5.4 that is causing it
>> to segfault during decoding.  The backtrace looks like this:
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread 46912496233216 (LWP 4091)]
>> 0x00000000004032c4 in _erl_free_term ()
>> (gdb) bt
>> #0  0x00000000004032c4 in _erl_free_term ()
>> #1  0x000000000040496b in erl_decode_it ()
>> #2  0x0000000000404937 in erl_decode_it ()
>> #3  0x0000000000404c93 in erl_decode_it ()
>> #4  0x0000000000404c93 in erl_decode_it ()
>> #5  0x0000000000405311 in erl_decode ()
>> #6  0x0000000000401b38 in main (argc=1, argv=0x7fff51ec7938) at
> badtest.c:28
>> The unfortunate part is that the way this large binary term is generated
>> cannot be done in any kind of sample code (it's being pulled off an
> external
>> database).
>>
>> Testing code:  http://jgray.la/erlang/erl_decode_segfault_test.tar.gz
>>
>> However, I have created a set of test files in C which recreate the
>> segfault.  I stored the binary in a flat file (as 'badbinary') and have a
>> testing program which reads it off disk and attempts to decode it.  To
> prove
>> the approach is sane (and that this segfault is related to something
> strange
>> about the decoding of this particular binary, not the size or general
> format
>> of the binary) there is a 'goodbinary' file and testing program for that.
>>
>> To use the test code:
>>
>> Untar/Ungzip the file.  You may need to edit the Makefile to fix the paths
>> to your erl_interface library.
>>
>> 'make' and then you can:
>>
>> ./badtest  (this reads 'badbinary' and attempts to decode, causes
> segfault)
>> ./goodtest (this reads 'goodbinary' and successfully decodes it)  [nearly
>> identical code to badtest.c but reads different file w/ different size]
>>
>> Also included is
>>
>> ./makegoodbin (a simple program that generates a large ETERM in an
> identical
>> format to the badbinary but contains duplicated binary data everywhere) 
>>
>>
>> Notes:
>>
>> * The marshaled binary erlang term being sent to C can be successfully
>> decoded/unmarshaled from within Erlang without a problem
>> * This is reproducible with many different large erlang terms generated
> from
>> our database queries.  'makegoodbin.c' creates a term identical in format
> to
>> those causing problems, however it does not have the random distribution
> of
>> binary sizes and content, and so I'm not able to reproduce the problem in
>> this way.
>> * The entire system, end-to-end including this decoding step, works
>> perfectly in most cases.  However when the data goes into the 100k+ range,
>> the segfaults start to happen.  That's why I created the 'makegoodbin'
> which
>> follows the same format.  Unfortunately that works even at sizes of >1MB
>> adding to the confusion of the problem.
>>
>>
>> Any help is appreciated.  Thanks.
>>
>> I apologize if this is a repost.  I never saw my original post hit the
> list
>> and did not receive any responses.
>>
>> Jonathan Gray
>> Streamy Inc.
>>
>> _______________________________________________
>> erlang-bugs mailing list
>> erlang-bugs@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-bugs
>>
> 
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
> 
> 



From jlist@REDACTED  Thu Jul 24 03:19:33 2008
From: jlist@REDACTED (Jonathan Gray)
Date: Wed, 23 Jul 2008 18:19:33 -0700
Subject: [erlang-bugs] Segfault in erl_interface attempting to decode
	certain large binaries
In-Reply-To: <4887D204.6020002@gmail.com>
References: <065901c8ec4e$fa744ff0$ef5cefd0$@com>
	<48870D3C.60606@gmail.com>	<06e801c8ecdb$cf94af40$6ebe0dc0$@com>
	<4887D204.6020002@gmail.com>
Message-ID: <077801c8ed2b$54cd5aa0$fe680fe0$@com>

Yeah I saw that tutorial.  Unfortunately it's based around a fixed format of
terms, thus doesn't make use of ei_get_type(), which is central to any kind
of generic decoder.

Regardless I did more extensive testing and it looks like I will be
implementing my own generic Erlang term decoder using ei.  Will post results
when I'm done.

Jonathan

-----Original Message-----
From: erlang-bugs-bounces@REDACTED [mailto:erlang-bugs-bounces@REDACTED]
On Behalf Of Serge Aleynikov
Sent: Wednesday, July 23, 2008 5:51 PM
To: Jonathan Gray
Cc: erlang-bugs@REDACTED
Subject: Re: [erlang-bugs] Segfault in erl_interface attempting to decode
certain large binaries

You can use the following tutorial to help you started with ei:
http://www.trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_progra
ms

I haven't seen any newer erl_encode/erl_decode implementations.  Frankly 
I've started doing that myself in C++ and got the recursive encoding 
part, but haven't done the decoder and had to put this project on hold 
due to other priorities.

Serge

Jonathan Gray wrote:
> Serge,
> 
> Sorry I was not more clear.  This is a port program with two remote
systems
> communicating via simple TCP packets (size header, buffer).
> 
> Thanks for the advice.  The documentation suggests that erl_interface is
> actually making use of the ei code.  I'm not sure if that's correct
because
> I see other resources on the web that say they are different (ei is new,
> erl_interface is old).  However the error does exist in a malloc() call so
> perhaps that can be avoided doing the term decodes myself?
> 
> Unfortunately what I need (already have working but segfaulting on a large
> decode) is a generic parser that converts ErlBinary into a number of
> different formats (JSON, Python, C struct tree) and vice versa.  The ETERM
> representation gave me an easy way to make recursive converters.
Recreating
> this generic behavior using ei seems like I'll be recreating functions of
> erl_interface and losing the ETERM representation so I'll need to rewrite
my
> converters in a completely different way. 
> 
> Rewriting it to go directly from binary would be a good thing in the long
> run, just not something I was planning on doing at this stage.  Has anyone
> ever written a newer generic erl_encode/erl_decode using the latest ei?
> 
> I will take a closer look at ei and report back.
> 
> Thanks for your help.
> 
> Jonathan
> 
> 
> -----Original Message-----
> From: erlang-bugs-bounces@REDACTED
[mailto:erlang-bugs-bounces@REDACTED]
> On Behalf Of Serge Aleynikov
> Sent: Wednesday, July 23, 2008 3:52 AM
> To: jlist@REDACTED
> Cc: erlang-bugs@REDACTED
> Subject: Re: [erlang-bugs] Segfault in erl_interface attempting to decode
> certain large binaries
> 
> Jonathan,
> 
> It's not clear from your description if you're using erl_interface to 
> build a driver or a port program.  While I don't have an answer to your 
> direct question, perhaps if you are writing a C port program you can use 
> ei instead of erl_interface, and if you are writing a driver, you can 
> use driver_output_term() / driver_send_term() and corresponding 
> ErlDrvTermData* structures to pass data to/from the emulator (*). This 
> is the fastest way to communicate with the emulator's port owner process.
> 
> (*) See: http://www.erlang.org/doc/man/erl_driver.html
> and also the source code of inet_drv.c in the distribution for various 
> LOAD_*() macros that simplify working with ErlDrvTermData structures.
> 
> Serge
> 
> 
> 
> 
> jlist@REDACTED wrote:
>> All,
>>
>> I have a TCP interface between an Erlang system and a C system.  Both
>> send/receive marshaled binary Erlang terms and I have not had any
problems
>> to date.
>>
>> Today I began doing some more serious testing with larger chunks of
binary
>> to be decoded in C.
>>
>> We ran into a bug (it seems) with erl_interface 3.5.5.4 that is causing
it
>> to segfault during decoding.  The backtrace looks like this:
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread 46912496233216 (LWP 4091)]
>> 0x00000000004032c4 in _erl_free_term ()
>> (gdb) bt
>> #0  0x00000000004032c4 in _erl_free_term ()
>> #1  0x000000000040496b in erl_decode_it ()
>> #2  0x0000000000404937 in erl_decode_it ()
>> #3  0x0000000000404c93 in erl_decode_it ()
>> #4  0x0000000000404c93 in erl_decode_it ()
>> #5  0x0000000000405311 in erl_decode ()
>> #6  0x0000000000401b38 in main (argc=1, argv=0x7fff51ec7938) at
> badtest.c:28
>> The unfortunate part is that the way this large binary term is generated
>> cannot be done in any kind of sample code (it's being pulled off an
> external
>> database).
>>
>> Testing code:  http://jgray.la/erlang/erl_decode_segfault_test.tar.gz
>>
>> However, I have created a set of test files in C which recreate the
>> segfault.  I stored the binary in a flat file (as 'badbinary') and have a
>> testing program which reads it off disk and attempts to decode it.  To
> prove
>> the approach is sane (and that this segfault is related to something
> strange
>> about the decoding of this particular binary, not the size or general
> format
>> of the binary) there is a 'goodbinary' file and testing program for that.
>>
>> To use the test code:
>>
>> Untar/Ungzip the file.  You may need to edit the Makefile to fix the
paths
>> to your erl_interface library.
>>
>> 'make' and then you can:
>>
>> ./badtest  (this reads 'badbinary' and attempts to decode, causes
> segfault)
>> ./goodtest (this reads 'goodbinary' and successfully decodes it)  [nearly
>> identical code to badtest.c but reads different file w/ different size]
>>
>> Also included is
>>
>> ./makegoodbin (a simple program that generates a large ETERM in an
> identical
>> format to the badbinary but contains duplicated binary data everywhere) 
>>
>>
>> Notes:
>>
>> * The marshaled binary erlang term being sent to C can be successfully
>> decoded/unmarshaled from within Erlang without a problem
>> * This is reproducible with many different large erlang terms generated
> from
>> our database queries.  'makegoodbin.c' creates a term identical in format
> to
>> those causing problems, however it does not have the random distribution
> of
>> binary sizes and content, and so I'm not able to reproduce the problem in
>> this way.
>> * The entire system, end-to-end including this decoding step, works
>> perfectly in most cases.  However when the data goes into the 100k+
range,
>> the segfaults start to happen.  That's why I created the 'makegoodbin'
> which
>> follows the same format.  Unfortunately that works even at sizes of >1MB
>> adding to the confusion of the problem.
>>
>>
>> Any help is appreciated.  Thanks.
>>
>> I apologize if this is a repost.  I never saw my original post hit the
> list
>> and did not receive any responses.
>>
>> Jonathan Gray
>> Streamy Inc.
>>
>> _______________________________________________
>> erlang-bugs mailing list
>> erlang-bugs@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-bugs
>>
> 
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
> 
> 

_______________________________________________
erlang-bugs mailing list
erlang-bugs@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-bugs



From saleyn@REDACTED  Thu Jul 24 03:33:40 2008
From: saleyn@REDACTED (Serge Aleynikov)
Date: Wed, 23 Jul 2008 21:33:40 -0400
Subject: [erlang-bugs] Segfault in erl_interface attempting to decode
 certain large binaries
In-Reply-To: <077801c8ed2b$54cd5aa0$fe680fe0$@com>
References: <065901c8ec4e$fa744ff0$ef5cefd0$@com> <48870D3C.60606@gmail.com>
	<06e801c8ecdb$cf94af40$6ebe0dc0$@com> <4887D204.6020002@gmail.com>
	<077801c8ed2b$54cd5aa0$fe680fe0$@com>
Message-ID: <4887DBF4.6050700@gmail.com>

The tutorial was kept simplistic on purpose.

Actually using ei for free-form terms is quite simple and only requires 
less than a day or so to get used to all flavors of ei_* functions.

Good luck!

Serge

Jonathan Gray wrote:
> Yeah I saw that tutorial.  Unfortunately it's based around a fixed format of
> terms, thus doesn't make use of ei_get_type(), which is central to any kind
> of generic decoder.
> 
> Regardless I did more extensive testing and it looks like I will be
> implementing my own generic Erlang term decoder using ei.  Will post results
> when I'm done.
> 
> Jonathan
> 
> -----Original Message-----
> From: erlang-bugs-bounces@REDACTED [mailto:erlang-bugs-bounces@REDACTED]
> On Behalf Of Serge Aleynikov
> Sent: Wednesday, July 23, 2008 5:51 PM
> To: Jonathan Gray
> Cc: erlang-bugs@REDACTED
> Subject: Re: [erlang-bugs] Segfault in erl_interface attempting to decode
> certain large binaries
> 
> You can use the following tutorial to help you started with ei:
> http://www.trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_progra
> ms
> 
> I haven't seen any newer erl_encode/erl_decode implementations.  Frankly 
> I've started doing that myself in C++ and got the recursive encoding 
> part, but haven't done the decoder and had to put this project on hold 
> due to other priorities.
> 
> Serge
> 
> Jonathan Gray wrote:
>> Serge,
>>
>> Sorry I was not more clear.  This is a port program with two remote
> systems
>> communicating via simple TCP packets (size header, buffer).
>>
>> Thanks for the advice.  The documentation suggests that erl_interface is
>> actually making use of the ei code.  I'm not sure if that's correct
> because
>> I see other resources on the web that say they are different (ei is new,
>> erl_interface is old).  However the error does exist in a malloc() call so
>> perhaps that can be avoided doing the term decodes myself?
>>
>> Unfortunately what I need (already have working but segfaulting on a large
>> decode) is a generic parser that converts ErlBinary into a number of
>> different formats (JSON, Python, C struct tree) and vice versa.  The ETERM
>> representation gave me an easy way to make recursive converters.
> Recreating
>> this generic behavior using ei seems like I'll be recreating functions of
>> erl_interface and losing the ETERM representation so I'll need to rewrite
> my
>> converters in a completely different way. 
>>
>> Rewriting it to go directly from binary would be a good thing in the long
>> run, just not something I was planning on doing at this stage.  Has anyone
>> ever written a newer generic erl_encode/erl_decode using the latest ei?
>>
>> I will take a closer look at ei and report back.
>>
>> Thanks for your help.
>>
>> Jonathan
>>
>>
>> -----Original Message-----
>> From: erlang-bugs-bounces@REDACTED
> [mailto:erlang-bugs-bounces@REDACTED]
>> On Behalf Of Serge Aleynikov
>> Sent: Wednesday, July 23, 2008 3:52 AM
>> To: jlist@REDACTED
>> Cc: erlang-bugs@REDACTED
>> Subject: Re: [erlang-bugs] Segfault in erl_interface attempting to decode
>> certain large binaries
>>
>> Jonathan,
>>
>> It's not clear from your description if you're using erl_interface to 
>> build a driver or a port program.  While I don't have an answer to your 
>> direct question, perhaps if you are writing a C port program you can use 
>> ei instead of erl_interface, and if you are writing a driver, you can 
>> use driver_output_term() / driver_send_term() and corresponding 
>> ErlDrvTermData* structures to pass data to/from the emulator (*). This 
>> is the fastest way to communicate with the emulator's port owner process.
>>
>> (*) See: http://www.erlang.org/doc/man/erl_driver.html
>> and also the source code of inet_drv.c in the distribution for various 
>> LOAD_*() macros that simplify working with ErlDrvTermData structures.
>>
>> Serge
>>
>>
>>
>>
>> jlist@REDACTED wrote:
>>> All,
>>>
>>> I have a TCP interface between an Erlang system and a C system.  Both
>>> send/receive marshaled binary Erlang terms and I have not had any
> problems
>>> to date.
>>>
>>> Today I began doing some more serious testing with larger chunks of
> binary
>>> to be decoded in C.
>>>
>>> We ran into a bug (it seems) with erl_interface 3.5.5.4 that is causing
> it
>>> to segfault during decoding.  The backtrace looks like this:
>>>
>>> Program received signal SIGSEGV, Segmentation fault.
>>> [Switching to Thread 46912496233216 (LWP 4091)]
>>> 0x00000000004032c4 in _erl_free_term ()
>>> (gdb) bt
>>> #0  0x00000000004032c4 in _erl_free_term ()
>>> #1  0x000000000040496b in erl_decode_it ()
>>> #2  0x0000000000404937 in erl_decode_it ()
>>> #3  0x0000000000404c93 in erl_decode_it ()
>>> #4  0x0000000000404c93 in erl_decode_it ()
>>> #5  0x0000000000405311 in erl_decode ()
>>> #6  0x0000000000401b38 in main (argc=1, argv=0x7fff51ec7938) at
>> badtest.c:28
>>> The unfortunate part is that the way this large binary term is generated
>>> cannot be done in any kind of sample code (it's being pulled off an
>> external
>>> database).
>>>
>>> Testing code:  http://jgray.la/erlang/erl_decode_segfault_test.tar.gz
>>>
>>> However, I have created a set of test files in C which recreate the
>>> segfault.  I stored the binary in a flat file (as 'badbinary') and have a
>>> testing program which reads it off disk and attempts to decode it.  To
>> prove
>>> the approach is sane (and that this segfault is related to something
>> strange
>>> about the decoding of this particular binary, not the size or general
>> format
>>> of the binary) there is a 'goodbinary' file and testing program for that.
>>>
>>> To use the test code:
>>>
>>> Untar/Ungzip the file.  You may need to edit the Makefile to fix the
> paths
>>> to your erl_interface library.
>>>
>>> 'make' and then you can:
>>>
>>> ./badtest  (this reads 'badbinary' and attempts to decode, causes
>> segfault)
>>> ./goodtest (this reads 'goodbinary' and successfully decodes it)  [nearly
>>> identical code to badtest.c but reads different file w/ different size]
>>>
>>> Also included is
>>>
>>> ./makegoodbin (a simple program that generates a large ETERM in an
>> identical
>>> format to the badbinary but contains duplicated binary data everywhere) 
>>>
>>>
>>> Notes:
>>>
>>> * The marshaled binary erlang term being sent to C can be successfully
>>> decoded/unmarshaled from within Erlang without a problem
>>> * This is reproducible with many different large erlang terms generated
>> from
>>> our database queries.  'makegoodbin.c' creates a term identical in format
>> to
>>> those causing problems, however it does not have the random distribution
>> of
>>> binary sizes and content, and so I'm not able to reproduce the problem in
>>> this way.
>>> * The entire system, end-to-end including this decoding step, works
>>> perfectly in most cases.  However when the data goes into the 100k+
> range,
>>> the segfaults start to happen.  That's why I created the 'makegoodbin'
>> which
>>> follows the same format.  Unfortunately that works even at sizes of >1MB
>>> adding to the confusion of the problem.
>>>
>>>
>>> Any help is appreciated.  Thanks.
>>>
>>> I apologize if this is a repost.  I never saw my original post hit the
>> list
>>> and did not receive any responses.
>>>
>>> Jonathan Gray
>>> Streamy Inc.
>>>
>>> _______________________________________________
>>> erlang-bugs mailing list
>>> erlang-bugs@REDACTED
>>> http://www.erlang.org/mailman/listinfo/erlang-bugs
>>>
>> _______________________________________________
>> erlang-bugs mailing list
>> erlang-bugs@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-bugs
>>
>>
> 
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
> 
> 



From sverker@REDACTED  Thu Jul 24 15:54:07 2008
From: sverker@REDACTED (Sverker Eriksson)
Date: Thu, 24 Jul 2008 15:54:07 +0200
Subject: [erlang-bugs] [BUG] OTP R12B-3: HTTP "space after headers" bug
 in	inet_drv.c
In-Reply-To: <6c2563b20807011606x7f00fa5bne75649182294bfba@mail.gmail.com>
References: <6c2563b20807011606x7f00fa5bne75649182294bfba@mail.gmail.com>
Message-ID: <4888897F.70209@erix.ericsson.se>

Thanks, for the bug-report.

The following simple patch seems to do the trick:

-if (SP(ptr2+1)) {
+if (SP(ptr2+1) && plen>2) {
    ptr1 = ptr2+1;
    len = n - plen;
}
else
    goto done;



Will be included in next release.

/Sverker, Erlang/OTP Ericsson


Edwin Fine wrote:
> First of all, many thanks to Claes Wikstrom for isolating and finding the
> module in which this bug lives (and for doing it so quickly). I originally
> thought this was a Yaws bug, and emailed Claes with the symptoms. It's quite
> repeatable (test code at end of email). He traced the bug to inet_drv.c and
> emailed me back with a small test program and details, and I offered to
> report the bug. Then I got curious as to what it could have been, so I did
> some debugging.
>
> The bug is in inet_drv.c. It will cause the end of the HTTP headers to be
> missed if the first character following the last CRLF (the one on its "own
> line") is a space or tab. This causes Yaws to time out after 30 seconds
> waiting for more header data that will never arrive. I am sure this will be
> true for httpd, too.
>
> The culprit (well, sort of; if you read on you will see why) is the
> following code on line 8324:
>
> *if (SP(ptr2+1)) {*
>     ptr1 = ptr2+1;
>     len = n - plen;
> }
> else
>     goto done;
>
> Consider the following HTTP POST data:
>
> "POST /invalid/url HTTP/1.1\r\n"
> "Connection: close\r\n"
> "Host: localhost:8000\r\n"
> "User-Agent: perl post\r\n"
> "Content-Length: 4\r\n"
> "Content-Type: text/xml; charset=utf-8\r\n"
> "\r\n"
> " postdata..." <-- Note this data starts with a space character
>
> When the inet_drv.c code gets to the last CRLF, it doesn't detect that it's
> the last CRLF (end of headers) and checks to see if the next character is a
> space or tab using the macro SP(ptr2+1). If there's any data following the
> last CRLF and it starts with a space or tab, the code as it stands now will
> think it's another header line and try to get more data, then time out when
> nothing arrives.
>
> What needed to be done was to check to see if the data (in that particular
> state) was an LF or CRLF standing on its own, and only check for a space
> following that if not. This check correctly detects the CRLF that terminates
> the header data. I added two lines of code to inet_drv.c to fix that error,
> and the Erlang test code (kindly supplied by Claes, and at the end of this
> email) now runs without error. I can't guarantee that the fix will work
> under all circumstances, and it needs to be tested thoroughly, but it's a
> starting point and illustrates the problem. (I would say that the HTTP
> header parsing in inet_drv.c probably could be beefed up a little).
>
> The patch to inet_drv.c is as follows (tabs may not be correct because I
> reformatted the code):
>
> 8318a8319,8323
>   
>> /* Test needed in case buffer is in form "\r\n\srequestdata" where \s is
>>     
> SP or TAB */
>   
>> if (((plen == 1) && NL(ptr)) || ((plen == 2) && CRNL(ptr)))
>>     goto done;
>>
>>     
>
> Claes's test code is here (slightly modified by me to make it easier to test
> the normal case and the "bug" case). You can run it as
> post:p(Host,Port,data) or post:p(Host,Port,bug).
>
> Regards,
> Edwin Fine
> ======================================
> -module(post).
> -compile(export_all).
>
> p() ->
>     p({127,0,0,1},8000,data).
>
> p(Host, Port, What) when What =:= bug; What =:= data ->
>    {ok, S} = gen_tcp:connect(Host, Port, [{active, true}]),
>    gen_tcp:send(S, select_data(What)),
>    recloop().
>
> recloop() ->
>    receive
>        X ->
>            io:format("GOT ~p~n", [X]),
>            recloop()
>    after 4000 ->
>            timeout
>    end.
>
> select_data(What) ->
>     case What of
>         bug ->
>             bug();
>         data ->
>             data()
>     end.
>
> data() ->
>    Msg = "dfoo",
>    H = "POST /invalid/url HTTP/1.1\r\n"
>        "Connection: close\r\n"
>        "Host: localhost:8000\r\n"
>        "User-Agent: perl post\r\n"
>        "Content-Length: 4\r\n"
>        "Content-Type: text/xml; charset=utf-8\r\n"
>        "\r\n",
>    H ++ Msg.
>
>
> %% space bug
> bug() ->
>    Msg = " foo",
>    H = "POST /invalid/url HTTP/1.1\r\n"
>        "Connection: close\r\n"
>        "Host: localhost:8000\r\n"
>        "User-Agent: perl post\r\n"
>        "Content-Length: 4\r\n"
>        "Content-Type: text/xml; charset=utf-8\r\n"
>        "\r\n",
>    H ++ Msg.
>
>
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs



From sverker@REDACTED  Thu Jul 24 18:35:00 2008
From: sverker@REDACTED (Sverker Eriksson)
Date: Thu, 24 Jul 2008 18:35:00 +0200
Subject: [erlang-bugs] Segfault in erl_interface attempting to decode
 certain large binaries
In-Reply-To: <065901c8ec4e$fa744ff0$ef5cefd0$@com>
References: <065901c8ec4e$fa744ff0$ef5cefd0$@com>
Message-ID: <4888AF34.6080307@erix.ericsson.se>

Hi Jonathan

I've looked at the test files you attached.

The file 'badbinary' is clearly broken. A lot of zeros is ending the file:

 > hexdump -C badbinary
:
0000ffb0 34 31 68 02 64 00 04 64 61 74 61 6d 00 00 00 08 |41h.d..datam....|
0000ffc0 54 65 72 72 65 6e 63 65 6a 6c 00 00 00 02 68 02 |Terrencejl....h.|
0000ffd0 64 00 03 72 6f 77 6d 00 00 00 24 38 37 32 61 36 |d..rowm...$872a6|
0000ffe0 62 62 61 2d 34 64 31 65 2d 34 34 64 34 2d 61 38 |bba-4d1e-44d4-a8|
0000fff0 61 62 2d 62 66 31 37 65 39 65 38 38 00 00 00 00 |ab-bf17e9e88....|
00010000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
0001e640 00 00 00 00 |....|
0001e644

The zeros are starting close to offset 0x10000 into the file.

The segfault happens when erl_interface finds the faulty format and 
fails to recover due to some bug. I will look at that.

I can't see how you could decode this data from within Erlang.

/Sverker, Erlang/OTP Ericsson


jlist@REDACTED wrote:
> All,
>
> I have a TCP interface between an Erlang system and a C system.  Both
> send/receive marshaled binary Erlang terms and I have not had any problems
> to date.
>
> Today I began doing some more serious testing with larger chunks of binary
> to be decoded in C.
>
> We ran into a bug (it seems) with erl_interface 3.5.5.4 that is causing it
> to segfault during decoding.  The backtrace looks like this:
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 46912496233216 (LWP 4091)]
> 0x00000000004032c4 in _erl_free_term ()
> (gdb) bt
> #0  0x00000000004032c4 in _erl_free_term ()
> #1  0x000000000040496b in erl_decode_it ()
> #2  0x0000000000404937 in erl_decode_it ()
> #3  0x0000000000404c93 in erl_decode_it ()
> #4  0x0000000000404c93 in erl_decode_it ()
> #5  0x0000000000405311 in erl_decode ()
> #6  0x0000000000401b38 in main (argc=1, argv=0x7fff51ec7938) at badtest.c:28
>
> The unfortunate part is that the way this large binary term is generated
> cannot be done in any kind of sample code (it?s being pulled off an external
> database).
>
> Testing code:  http://jgray.la/erlang/erl_decode_segfault_test.tar.gz
>
> However, I have created a set of test files in C which recreate the
> segfault.  I stored the binary in a flat file (as ?badbinary?) and have a
> testing program which reads it off disk and attempts to decode it.  To prove
> the approach is sane (and that this segfault is related to something strange
> about the decoding of this particular binary, not the size or general format
> of the binary) there is a ?goodbinary? file and testing program for that.
>
> To use the test code:
>
> Untar/Ungzip the file.  You may need to edit the Makefile to fix the paths
> to your erl_interface library.
>
> ?make? and then you can:
>
> ./badtest  (this reads ?badbinary? and attempts to decode, causes segfault)
> ./goodtest (this reads ?goodbinary? and successfully decodes it)  [nearly
> identical code to badtest.c but reads different file w/ different size]
>
> Also included is
>
> ./makegoodbin (a simple program that generates a large ETERM in an identical
> format to the badbinary but contains duplicated binary data everywhere) 
>
>
> Notes:
>
> * The marshaled binary erlang term being sent to C can be successfully
> decoded/unmarshaled from within Erlang without a problem
> * This is reproducible with many different large erlang terms generated from
> our database queries.  ?makegoodbin.c? creates a term identical in format to
> those causing problems, however it does not have the random distribution of
> binary sizes and content, and so I?m not able to reproduce the problem in
> this way.
> * The entire system, end-to-end including this decoding step, works
> perfectly in most cases.  However when the data goes into the 100k+ range,
> the segfaults start to happen.  That?s why I created the ?makegoodbin? which
> follows the same format.  Unfortunately that works even at sizes of >1MB
> adding to the confusion of the problem.
>
>
> Any help is appreciated.  Thanks.
>
> I apologize if this is a repost.  I never saw my original post hit the list
> and did not receive any responses.
>
> Jonathan Gray
> Streamy Inc.
>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
>   



From jlist@REDACTED  Thu Jul 24 19:52:52 2008
From: jlist@REDACTED (Jonathan Gray)
Date: Thu, 24 Jul 2008 10:52:52 -0700
Subject: [erlang-bugs] Segfault in erl_interface attempting to decode
	certain large binaries
In-Reply-To: <4888AF34.6080307@erix.ericsson.se>
References: <065901c8ec4e$fa744ff0$ef5cefd0$@com>
	<4888AF34.6080307@erix.ericsson.se>
Message-ID: <086001c8edb6$189ef300$49dcd900$@com>

Sverker,

I apologize for not looking directly at the binary to see if there were any
strange issues.

Looks like I have my C server not blocking until it receives the entire
sequence.  That will certainly clear this up.

Thanks for your help.  I am nearly complete with the ei based parser so will
probably go forward with that since it will be more efficient than
converting to ETERM and then parsing.  So this error on my part may be a
blessing in disguise (or I will keep telling myself that)  :)

Thanks again!

Jonathan

-----Original Message-----
From: erlang-bugs-bounces@REDACTED [mailto:erlang-bugs-bounces@REDACTED]
On Behalf Of Sverker Eriksson
Sent: Thursday, July 24, 2008 9:35 AM
To: jlist@REDACTED
Cc: erlang-bugs@REDACTED
Subject: Re: [erlang-bugs] Segfault in erl_interface attempting to decode
certain large binaries

Hi Jonathan

I've looked at the test files you attached.

The file 'badbinary' is clearly broken. A lot of zeros is ending the file:

 > hexdump -C badbinary
:
0000ffb0 34 31 68 02 64 00 04 64 61 74 61 6d 00 00 00 08 |41h.d..datam....|
0000ffc0 54 65 72 72 65 6e 63 65 6a 6c 00 00 00 02 68 02 |Terrencejl....h.|
0000ffd0 64 00 03 72 6f 77 6d 00 00 00 24 38 37 32 61 36 |d..rowm...$872a6|
0000ffe0 62 62 61 2d 34 64 31 65 2d 34 34 64 34 2d 61 38 |bba-4d1e-44d4-a8|
0000fff0 61 62 2d 62 66 31 37 65 39 65 38 38 00 00 00 00 |ab-bf17e9e88....|
00010000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
0001e640 00 00 00 00 |....|
0001e644

The zeros are starting close to offset 0x10000 into the file.

The segfault happens when erl_interface finds the faulty format and 
fails to recover due to some bug. I will look at that.

I can't see how you could decode this data from within Erlang.

/Sverker, Erlang/OTP Ericsson


jlist@REDACTED wrote:
> All,
>
> I have a TCP interface between an Erlang system and a C system.  Both
> send/receive marshaled binary Erlang terms and I have not had any problems
> to date.
>
> Today I began doing some more serious testing with larger chunks of binary
> to be decoded in C.
>
> We ran into a bug (it seems) with erl_interface 3.5.5.4 that is causing it
> to segfault during decoding.  The backtrace looks like this:
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 46912496233216 (LWP 4091)]
> 0x00000000004032c4 in _erl_free_term ()
> (gdb) bt
> #0  0x00000000004032c4 in _erl_free_term ()
> #1  0x000000000040496b in erl_decode_it ()
> #2  0x0000000000404937 in erl_decode_it ()
> #3  0x0000000000404c93 in erl_decode_it ()
> #4  0x0000000000404c93 in erl_decode_it ()
> #5  0x0000000000405311 in erl_decode ()
> #6  0x0000000000401b38 in main (argc=1, argv=0x7fff51ec7938) at
badtest.c:28
>
> The unfortunate part is that the way this large binary term is generated
> cannot be done in any kind of sample code (it's being pulled off an
external
> database).
>
> Testing code:  http://jgray.la/erlang/erl_decode_segfault_test.tar.gz
>
> However, I have created a set of test files in C which recreate the
> segfault.  I stored the binary in a flat file (as 'badbinary') and have a
> testing program which reads it off disk and attempts to decode it.  To
prove
> the approach is sane (and that this segfault is related to something
strange
> about the decoding of this particular binary, not the size or general
format
> of the binary) there is a 'goodbinary' file and testing program for that.
>
> To use the test code:
>
> Untar/Ungzip the file.  You may need to edit the Makefile to fix the paths
> to your erl_interface library.
>
> 'make' and then you can:
>
> ./badtest  (this reads 'badbinary' and attempts to decode, causes
segfault)
> ./goodtest (this reads 'goodbinary' and successfully decodes it)  [nearly
> identical code to badtest.c but reads different file w/ different size]
>
> Also included is
>
> ./makegoodbin (a simple program that generates a large ETERM in an
identical
> format to the badbinary but contains duplicated binary data everywhere) 
>
>
> Notes:
>
> * The marshaled binary erlang term being sent to C can be successfully
> decoded/unmarshaled from within Erlang without a problem
> * This is reproducible with many different large erlang terms generated
from
> our database queries.  'makegoodbin.c' creates a term identical in format
to
> those causing problems, however it does not have the random distribution
of
> binary sizes and content, and so I'm not able to reproduce the problem in
> this way.
> * The entire system, end-to-end including this decoding step, works
> perfectly in most cases.  However when the data goes into the 100k+ range,
> the segfaults start to happen.  That's why I created the 'makegoodbin'
which
> follows the same format.  Unfortunately that works even at sizes of >1MB
> adding to the confusion of the problem.
>
>
> Any help is appreciated.  Thanks.
>
> I apologize if this is a repost.  I never saw my original post hit the
list
> and did not receive any responses.
>
> Jonathan Gray
> Streamy Inc.
>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
>   

_______________________________________________
erlang-bugs mailing list
erlang-bugs@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-bugs



From sverker@REDACTED  Mon Jul 28 11:21:46 2008
From: sverker@REDACTED (Sverker Eriksson)
Date: Mon, 28 Jul 2008 11:21:46 +0200
Subject: [erlang-bugs] R12B3 epmd debug message typo
In-Reply-To: <20080721071308.GA39531@k2r.org>
References: <20080721071308.GA39531@k2r.org>
Message-ID: <488D8FAA.5040800@erix.ericsson.se>

Has been corrected.

Thanks
/Sverker, Erlang/OTP Ericsson


Kenji Rikitake wrote:
> I've found in R12B3 the word "epmd" are typed in as "empd".
> This is confusing and should be fixed.
> A patch included in this message.
>
> Kenji Rikitake
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs



From sverker@REDACTED  Mon Jul 28 12:11:03 2008
From: sverker@REDACTED (Sverker Eriksson)
Date: Mon, 28 Jul 2008 12:11:03 +0200
Subject: [erlang-bugs] regexp: no pattern matches empty string
In-Reply-To: <36278dc50807201102o32fb2481r4350b02b078cf1ea@mail.gmail.com>
References: <36278dc50807152240s75ebabd4tc7e23d7c5f2f63ea@mail.gmail.com>
	<36278dc50807201102o32fb2481r4350b02b078cf1ea@mail.gmail.com>
Message-ID: <488D9B37.8010203@erix.ericsson.se>

Taisuke Hori wrote:
> Hello,
>
> I suspect a bug in regexp. I tried to match some patterns to a empty
> list, but they ware not matched.
>
> $ erl -version
> Erlang (ASYNC_THREADS,HIPE) (BEAM) emulator version 5.5.5
> $ erl
> Eshell V5.5.5  (abort with ^G)
> 1> regexp:match("", "").
> nomatch
>   
Is an empty string really a valid regular expression? The new 
experimental module "re" in R12b-3, does not allow that:

1> re:run("", "").
** exception error: bad argument
     in function  re:run/2
        called as re:run([],[])

> 2> regexp:match("", ".*").
> nomatch
>   
This looks like a bug to me or at least like a strange behavior. "re" 
handles it well:

2> re:run("", ".*").
{match,[{0,0}]}


/Sverker, Erlang/OTP Ericsson



From sverker@REDACTED  Mon Jul 28 17:57:43 2008
From: sverker@REDACTED (Sverker Eriksson)
Date: Mon, 28 Jul 2008 17:57:43 +0200
Subject: [erlang-bugs] crypto module incorrectly linked with DYNAMIC_SSL
In-Reply-To: <20080712111354.2555c57a@gentoo.org>
References: <20080712111354.2555c57a@gentoo.org>
Message-ID: <488DEC77.1010808@erix.ericsson.se>

A somewhat late answer and maybe I have misunderstood the problem, but...

Default linking of ssl is static. According to config.log you are not 
using --enable-dynamic-ssl-lib to tell configure to use dynamic linking.

/Sverker, Erlang/OTP Ericsson


Christian Faulhammer wrote:
> * PGP Signed by an unknown key
>
> Hi,
>
> a user of Gentoo Linux reported the following bug via our bug tracker:
> <URL:http://bugs.gentoo.org/show_bug.cgi?id=231468>.  Summary is:
>
> Erlang (BEAM) emulator version 5.6.3 [source] [smp:4] [async-threads:0]
>
> Eshell V5.6.3  (abort with ^G)
> 1> crypto:start().
> sh: line 0: exec: crypto_drv: not found
>                                       
> {error,{shutdown,{crypto_app,start,[normal,[]]}}}
>
> We set SSL_DYNAMIC to yes duing build to have the system's libraries
> used for linking.  Unfortunately this does not work.  
>
> $ ldd /usr/lib/erlang/lib/crypto-1.5.2/priv/lib/crypto_drv.so
>         linux-gate.so.1 =>  (0x5376b000)
>         libc.so.6 => /lib/libc.so.6 (0x535b4000)
>         /lib/ld-linux.so.2 (0x5376c000)
>
> For his config.log file see the mentioned bug report.  Thanks for your
> support.
>
> V-Li
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs
>
>
> * PGP Signed by an unknown key
> * text/plain body
> * Unknown Key
> * 0x2B859DE3
>   



From ingela@REDACTED  Tue Jul 29 11:15:52 2008
From: ingela@REDACTED (Ingela Anderton Andin)
Date: Tue, 29 Jul 2008 11:15:52 +0200
Subject: [erlang-bugs] Incorrect or misleading inets/http_server error
 message
In-Reply-To: <6c2563b20806301456j5276f073q522ddb29470e2474@mail.gmail.com>
References: <6c2563b20806301456j5276f073q522ddb29470e2474@mail.gmail.com>
Message-ID: <488EDFC8.5050603@erix.ericsson.se>

Hi Edwin!

Well yes in this particular case it becomes a bit confusing I agree.  It 
is a type check that has fail but this is obviously
a special case of failure so I changed it so it will become 
*{wrong_type, {non_existing, document_root,"/tmp/htdocs"}}.
The type check acctualy checks if  "/tmp/htdocs" is a directory and not 
only that it is a string on the correct format. So the type here is 
"directory"
and not "string". But yes I can see that people might get confused. I 
think this will be the best solution removing the wrong_type in this case
will make the code cluttered of special cases in more than one place, 
now I just handle the special case where it appeares and the higer level 
code does not need to care about such details, and also it is logical 
type error, the given parameter was not a valid directory.

Regards - Ingela Erlang/OTP, Ericsson


*Edwin Fine wrote:
> Taking the example out of the document at 
> http://www.erlang.org/doc/apps/inets/http_server.html:
>
> 2> {ok, Pid} = inets:start(httpd, [{port, 0},
> 2>       {server_name,"httpd_test"}, {server_root,"/tmp"},
> 2>       {document_root,"/tmp/htdocs"}, {bind_address, "localhost"}]).
>
> =ERROR REPORT==== 30-Jun-2008::17:47:18 ===
> Failed initiating web server:
> undefined
> *{wrong_type,{document_root,"/tmp/htdocs"}}*
>
> ** exception error: no match of right hand side value 
> {error,{shutdown,{child,undefined,
>                                                                               
> {httpd_instance_sup,{0,0,0,0,0,0,0,1},59104},
>                                                                               
> {httpd_instance_sup,start_link,
>                                                                                                   
> [[{port,59104},
>                                                                                                     
> {bind_address,{0,0,0,0,0,0,0,1}},
>                                                                                                     
> {server_name,"httpd_test"},
>                                                                                                     
> {server_root,"/tmp"},
>                                                                                                     
> {document_root,"/tmp/htdocs"}],
>                                                                                                    
> 15000,
>                                                                                                    
> {<0.50.0>,#Port<0.126>},
>                                                                                                    
> []]},
>                                                                               
> permanent,infinity,supervisor,
>                                                                               
> [httpd_instance_sup]}}}
>
> The actual problem was that the "/tmp/htdocs" directory did not exist. 
> I find having an error message of undefined 
> *{wrong_type,{document_root,"/tmp/htdocs"}} *to be very misleading. I 
> read it as "the value corresponding to document_root [i.e. 
> "/tmp/htdocs", of type string] is of the wrong type". If not a string, 
> what should it be, I wondered?
>
> When I created the directory and tried again, the message went away. I 
> would have expected to see something like 
> {does_not_exist,{document_root,"/tmp/htdocs"}}.
>
> 3> {ok, Pid} = inets:start(httpd, [{port, 0},                        
> 3>       {server_name,"httpd_test"}, {server_root,"/tmp"},
> 3>       {document_root,"/tmp/htdocs"}, {bind_address, "localhost"}]).
> {ok,<0.58.0>}
> 4>
>
> Regards,
> Edwin Fine
> -- 
> The great enemy of the truth is very often not the lie -- deliberate, 
> contrived and dishonest, but the myth, persistent, persuasive, and 
> unrealistic. Belief in myths allows the comfort of opinion without the 
> discomfort of thought.
> John F. Kennedy 35th president of US 1961-1963 (1917 - 1963)
> ------------------------------------------------------------------------
>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-bugs



From ingela@REDACTED  Tue Jul 29 12:05:56 2008
From: ingela@REDACTED (Ingela Anderton Andin)
Date: Tue, 29 Jul 2008 12:05:56 +0200
Subject: [erlang-bugs] inets bad info message
In-Reply-To: <d3520a2f0806231552r4ec8a95ag68cd64fc3748e130@mail.gmail.com>
References: <d3520a2f0806231552r4ec8a95ag68cd64fc3748e130@mail.gmail.com>
Message-ID: <488EEB84.9090905@erix.ericsson.se>


Anders Nygren wrote:
> Hi
> When giving a bad HTTPOption in http:request I get this info message.
> I assume that I was supposed to be formatted differently.
>
> == 2008-06-23 17:04:43 === INFO =========================== <0.239.0> ==
>    {"Invalid option ignored ~p~n",[{http_timeout,20000}]}
>
>   

Yes that was intended to be a little differently formatted,  I have now 
changed it so it will now read:

== 2008-06-23 17:04:43 === INFO =========================== <0.239.0> ==
Invalid option {http_timeout, 20000} ignored


Regards Ingela - Erlang/OTP, Ericsson




From erlang-questions_efine@REDACTED  Tue Jul 29 17:28:59 2008
From: erlang-questions_efine@REDACTED (Edwin Fine)
Date: Tue, 29 Jul 2008 11:28:59 -0400
Subject: [erlang-bugs] Incorrect or misleading inets/http_server error
	message
In-Reply-To: <488EDFC8.5050603@erix.ericsson.se>
References: <6c2563b20806301456j5276f073q522ddb29470e2474@mail.gmail.com>
	<488EDFC8.5050603@erix.ericsson.se>
Message-ID: <6c2563b20807290828g3ecb70f5x382f6286d8c941f9@mail.gmail.com>

Thanks, Ingela, your solution looks good, but I find it still a little bit
misleading. You see, what was provided was actually a directory (in string
format). The fact that the directory does not exist has nothing to do with
its type, really. What it has to do with is validation. Maybe wrong_type
would read better as validation_failed? That covers both type checks and
existence checks and any other kind of checks of this sort that may come up
in future.

Regards,
Edwin

On Tue, Jul 29, 2008 at 5:15 AM, Ingela Anderton Andin <
ingela@REDACTED> wrote:

> Hi Edwin!
>
> Well yes in this particular case it becomes a bit confusing I agree.  It is
> a type check that has fail but this is obviously
> a special case of failure so I changed it so it will become *{wrong_type,
> {non_existing, document_root,"/tmp/htdocs"}}.
> The type check acctualy checks if  "/tmp/htdocs" is a directory and not
> only that it is a string on the correct format. So the type here is
> "directory"
> and not "string". But yes I can see that people might get confused. I think
> this will be the best solution removing the wrong_type in this case
> will make the code cluttered of special cases in more than one place, now I
> just handle the special case where it appeares and the higer level code does
> not need to care about such details, and also it is logical type error, the
> given parameter was not a valid directory.
>
> Regards - Ingela Erlang/OTP, Ericsson
>
>
> *Edwin Fine wrote:
>
>> Taking the example out of the document at
>> http://www.erlang.org/doc/apps/inets/http_server.html:
>>
>> 2> {ok, Pid} = inets:start(httpd, [{port, 0},
>> 2>       {server_name,"httpd_test"}, {server_root,"/tmp"},
>> 2>       {document_root,"/tmp/htdocs"}, {bind_address, "localhost"}]).
>>
>> =ERROR REPORT==== 30-Jun-2008::17:47:18 ===
>> Failed initiating web server:
>> undefined
>> *{wrong_type,{document_root,"/tmp/htdocs"}}*
>>
>> ** exception error: no match of right hand side value
>> {error,{shutdown,{child,undefined,
>>
>>    {httpd_instance_sup,{0,0,0,0,0,0,0,1},59104},
>>
>>    {httpd_instance_sup,start_link,
>>
>>                        [[{port,59104},
>>
>>                          {bind_address,{0,0,0,0,0,0,0,1}},
>>
>>                          {server_name,"httpd_test"},
>>
>>                          {server_root,"/tmp"},
>>
>>                          {document_root,"/tmp/htdocs"}],
>>
>>                         15000,
>>
>>                         {<0.50.0>,#Port<0.126>},
>>
>>                         []]},
>>
>>    permanent,infinity,supervisor,
>>
>>    [httpd_instance_sup]}}}
>>
>> The actual problem was that the "/tmp/htdocs" directory did not exist. I
>> find having an error message of undefined
>> *{wrong_type,{document_root,"/tmp/htdocs"}} *to be very misleading. I read
>> it as "the value corresponding to document_root [i.e. "/tmp/htdocs", of type
>> string] is of the wrong type". If not a string, what should it be, I
>> wondered?
>>
>> When I created the directory and tried again, the message went away. I
>> would have expected to see something like
>> {does_not_exist,{document_root,"/tmp/htdocs"}}.
>>
>> 3> {ok, Pid} = inets:start(httpd, [{port, 0},                        3>
>>     {server_name,"httpd_test"}, {server_root,"/tmp"},
>> 3>       {document_root,"/tmp/htdocs"}, {bind_address, "localhost"}]).
>> {ok,<0.58.0>}
>> 4>
>>
>> Regards,
>> Edwin Fine
>> --
>> The great enemy of the truth is very often not the lie -- deliberate,
>> contrived and dishonest, but the myth, persistent, persuasive, and
>> unrealistic. Belief in myths allows the comfort of opinion without the
>> discomfort of thought.
>> John F. Kennedy 35th president of US 1961-1963 (1917 - 1963)
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> erlang-bugs mailing list
>> erlang-bugs@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-bugs
>>
>
>
>


-- 
For every expert there is an equal and opposite expert - Arthur C. Clarke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080729/95c277b5/attachment.htm>

From ingela@REDACTED  Wed Jul 30 08:35:44 2008
From: ingela@REDACTED (Ingela Anderton Andin)
Date: Wed, 30 Jul 2008 08:35:44 +0200
Subject: [erlang-bugs] Incorrect or misleading inets/http_server error
 message
In-Reply-To: <6c2563b20807290828g3ecb70f5x382f6286d8c941f9@mail.gmail.com>
References: <6c2563b20806301456j5276f073q522ddb29470e2474@mail.gmail.com>	
	<488EDFC8.5050603@erix.ericsson.se>
	<6c2563b20807290828g3ecb70f5x382f6286d8c941f9@mail.gmail.com>
Message-ID: <48900BC0.3080307@erix.ericsson.se>

Hi Edwin,

well I suppose it is a little too much of a question of how you define 
type, and maybe your reasoning is more the common way to reason than 
mine. I think of "type" in a quite abstract way.  But I do not want to 
argue on what is more right or wrong, I think it is fair to choose 
another  wording  that  will be more clear to
all.  I suggest  {invalid_option, {non_existing, 
{document_root,"/tmp/htdocs"}}}. How do you feel about that?

Regards - Ingela Erlang/OTP, Ericsson

Edwin Fine wrote:
> Thanks, Ingela, your solution looks good, but I find it still a little 
> bit misleading. You see, what was provided was actually a directory 
> (in string format). The fact that the directory does not exist has 
> nothing to do with its type, really. What it has to do with is 
> validation. Maybe wrong_type would read better as validation_failed? 
> That covers both type checks and existence checks and any other kind 
> of checks of this sort that may come up in future.
>
> Regards,
> Edwin
>
> On Tue, Jul 29, 2008 at 5:15 AM, Ingela Anderton Andin 
> <ingela@REDACTED <mailto:ingela@REDACTED>> wrote:
>
>     Hi Edwin!
>
>     Well yes in this particular case it becomes a bit confusing I
>     agree.  It is a type check that has fail but this is obviously
>     a special case of failure so I changed it so it will become
>     *{wrong_type, {non_existing, document_root,"/tmp/htdocs"}}.
>     The type check acctualy checks if  "/tmp/htdocs" is a directory
>     and not only that it is a string on the correct format. So the
>     type here is "directory"
>     and not "string". But yes I can see that people might get
>     confused. I think this will be the best solution removing the
>     wrong_type in this case
>     will make the code cluttered of special cases in more than one
>     place, now I just handle the special case where it appeares and
>     the higer level code does not need to care about such details, and
>     also it is logical type error, the given parameter was not a valid
>     directory.
>
>     Regards - Ingela Erlang/OTP, Ericsson
>
>
>     *Edwin Fine wrote:
>
>         Taking the example out of the document at
>         http://www.erlang.org/doc/apps/inets/http_server.html:
>
>         2> {ok, Pid} = inets:start(httpd, [{port, 0},
>         2>       {server_name,"httpd_test"}, {server_root,"/tmp"},
>         2>       {document_root,"/tmp/htdocs"}, {bind_address,
>         "localhost"}]).
>
>         =ERROR REPORT==== 30-Jun-2008::17:47:18 ===
>         Failed initiating web server:
>         undefined
>         *{wrong_type,{document_root,"/tmp/htdocs"}}*
>
>         ** exception error: no match of right hand side value
>         {error,{shutdown,{child,undefined,
>                                                                      
>                        {httpd_instance_sup,{0,0,0,0,0,0,0,1},59104},
>                                                                      
>                        {httpd_instance_sup,start_link,
>                                                                      
>                                            [[{port,59104},
>                                                                      
>                                            
>          {bind_address,{0,0,0,0,0,0,0,1}},
>                                                                      
>                                              {server_name,"httpd_test"},
>                                                                      
>                                              {server_root,"/tmp"},
>                                                                      
>                                            
>          {document_root,"/tmp/htdocs"}],
>                                                                      
>                                             15000,
>                                                                      
>                                             {<0.50.0>,#Port<0.126>},
>                                                                      
>                                             []]},
>                                                                      
>                        permanent,infinity,supervisor,
>                                                                      
>                        [httpd_instance_sup]}}}
>
>         The actual problem was that the "/tmp/htdocs" directory did
>         not exist. I find having an error message of undefined
>         *{wrong_type,{document_root,"/tmp/htdocs"}} *to be very
>         misleading. I read it as "the value corresponding to
>         document_root [i.e. "/tmp/htdocs", of type string] is of the
>         wrong type". If not a string, what should it be, I wondered?
>
>         When I created the directory and tried again, the message went
>         away. I would have expected to see something like
>         {does_not_exist,{document_root,"/tmp/htdocs"}}.
>
>         3> {ok, Pid} = inets:start(httpd, [{port, 0},                
>                3>       {server_name,"httpd_test"}, {server_root,"/tmp"},
>         3>       {document_root,"/tmp/htdocs"}, {bind_address,
>         "localhost"}]).
>         {ok,<0.58.0>}
>         4>
>
>         Regards,
>         Edwin Fine
>         -- 
>         The great enemy of the truth is very often not the lie --
>         deliberate, contrived and dishonest, but the myth, persistent,
>         persuasive, and unrealistic. Belief in myths allows the
>         comfort of opinion without the discomfort of thought.
>         John F. Kennedy 35th president of US 1961-1963 (1917 - 1963)
>         ------------------------------------------------------------------------
>
>         _______________________________________________
>         erlang-bugs mailing list
>         erlang-bugs@REDACTED <mailto:erlang-bugs@REDACTED>
>         http://www.erlang.org/mailman/listinfo/erlang-bugs
>
>
>
>
>
>
> -- 
> For every expert there is an equal and opposite expert - Arthur C. Clarke



From erlang-questions_efine@REDACTED  Wed Jul 30 16:24:07 2008
From: erlang-questions_efine@REDACTED (Edwin Fine)
Date: Wed, 30 Jul 2008 10:24:07 -0400
Subject: [erlang-bugs] Incorrect or misleading inets/http_server error
	message
In-Reply-To: <48900BC0.3080307@erix.ericsson.se>
References: <6c2563b20806301456j5276f073q522ddb29470e2474@mail.gmail.com>
	<488EDFC8.5050603@erix.ericsson.se>
	<6c2563b20807290828g3ecb70f5x382f6286d8c941f9@mail.gmail.com>
	<48900BC0.3080307@erix.ericsson.se>
Message-ID: <6c2563b20807300724v7483965due54cd4160c6b0995@mail.gmail.com>

Hi Ingela, I think that what you suggest is very clear. Thank you.

On Wed, Jul 30, 2008 at 2:35 AM, Ingela Anderton Andin <
ingela@REDACTED> wrote:

> Hi Edwin,
>
> well I suppose it is a little too much of a question of how you define
> type, and maybe your reasoning is more the common way to reason than mine. I
> think of "type" in a quite abstract way.  But I do not want to argue on what
> is more right or wrong, I think it is fair to choose another  wording  that
>  will be more clear to
> all.  I suggest  {invalid_option, {non_existing,
> {document_root,"/tmp/htdocs"}}}. How do you feel about that?
>
> Regards - Ingela Erlang/OTP, Ericsson
>
> Edwin Fine wrote:
>
>> Thanks, Ingela, your solution looks good, but I find it still a little bit
>> misleading. You see, what was provided was actually a directory (in string
>> format). The fact that the directory does not exist has nothing to do with
>> its type, really. What it has to do with is validation. Maybe wrong_type
>> would read better as validation_failed? That covers both type checks and
>> existence checks and any other kind of checks of this sort that may come up
>> in future.
>>
>> Regards,
>> Edwin
>>
>> On Tue, Jul 29, 2008 at 5:15 AM, Ingela Anderton Andin <
>> ingela@REDACTED <mailto:ingela@REDACTED>> wrote:
>>
>>    Hi Edwin!
>>
>>    Well yes in this particular case it becomes a bit confusing I
>>    agree.  It is a type check that has fail but this is obviously
>>    a special case of failure so I changed it so it will become
>>    *{wrong_type, {non_existing, document_root,"/tmp/htdocs"}}.
>>    The type check acctualy checks if  "/tmp/htdocs" is a directory
>>    and not only that it is a string on the correct format. So the
>>    type here is "directory"
>>    and not "string". But yes I can see that people might get
>>    confused. I think this will be the best solution removing the
>>    wrong_type in this case
>>    will make the code cluttered of special cases in more than one
>>    place, now I just handle the special case where it appeares and
>>    the higer level code does not need to care about such details, and
>>    also it is logical type error, the given parameter was not a valid
>>    directory.
>>
>>    Regards - Ingela Erlang/OTP, Ericsson
>>
>>
>>    *Edwin Fine wrote:
>>
>>        Taking the example out of the document at
>>        http://www.erlang.org/doc/apps/inets/http_server.html:
>>
>>        2> {ok, Pid} = inets:start(httpd, [{port, 0},
>>        2>       {server_name,"httpd_test"}, {server_root,"/tmp"},
>>        2>       {document_root,"/tmp/htdocs"}, {bind_address,
>>        "localhost"}]).
>>
>>        =ERROR REPORT==== 30-Jun-2008::17:47:18 ===
>>        Failed initiating web server:
>>        undefined
>>        *{wrong_type,{document_root,"/tmp/htdocs"}}*
>>
>>        ** exception error: no match of right hand side value
>>        {error,{shutdown,{child,undefined,
>>
>>                 {httpd_instance_sup,{0,0,0,0,0,0,0,1},59104},
>>
>>                 {httpd_instance_sup,start_link,
>>
>>                                     [[{port,59104},
>>
>>
>> {bind_address,{0,0,0,0,0,0,0,1}},
>>
>>                                       {server_name,"httpd_test"},
>>
>>                                       {server_root,"/tmp"},
>>
>>                                             {document_root,"/tmp/htdocs"}],
>>
>>                                      15000,
>>
>>                                      {<0.50.0>,#Port<0.126>},
>>
>>                                      []]},
>>
>>                 permanent,infinity,supervisor,
>>
>>                 [httpd_instance_sup]}}}
>>
>>        The actual problem was that the "/tmp/htdocs" directory did
>>        not exist. I find having an error message of undefined
>>        *{wrong_type,{document_root,"/tmp/htdocs"}} *to be very
>>        misleading. I read it as "the value corresponding to
>>        document_root [i.e. "/tmp/htdocs", of type string] is of the
>>        wrong type". If not a string, what should it be, I wondered?
>>
>>        When I created the directory and tried again, the message went
>>        away. I would have expected to see something like
>>        {does_not_exist,{document_root,"/tmp/htdocs"}}.
>>
>>        3> {ok, Pid} = inets:start(httpd, [{port, 0},
>>         3>       {server_name,"httpd_test"}, {server_root,"/tmp"},
>>        3>       {document_root,"/tmp/htdocs"}, {bind_address,
>>        "localhost"}]).
>>        {ok,<0.58.0>}
>>        4>
>>
>>        Regards,
>>        Edwin Fine
>>        --        The great enemy of the truth is very often not the lie --
>>        deliberate, contrived and dishonest, but the myth, persistent,
>>        persuasive, and unrealistic. Belief in myths allows the
>>        comfort of opinion without the discomfort of thought.
>>        John F. Kennedy 35th president of US 1961-1963 (1917 - 1963)
>>
>>  ------------------------------------------------------------------------
>>
>>        _______________________________________________
>>        erlang-bugs mailing list
>>        erlang-bugs@REDACTED <mailto:erlang-bugs@REDACTED>
>>        http://www.erlang.org/mailman/listinfo/erlang-bugs
>>
>>
>>
>>
>>
>>
>> --
>> For every expert there is an equal and opposite expert - Arthur C. Clarke
>>
>
>
>


-- 
For every expert there is an equal and opposite expert - Arthur C. Clarke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080730/70f4bb9c/attachment.htm>

From jlist@REDACTED  Sat Jul 19 03:29:31 2008
From: jlist@REDACTED (Jonathan Gray)
Date: Sat, 19 Jul 2008 01:29:31 -0000
Subject: [erlang-bugs] Segfault in erl_interface attempting to decode
	certain large binaries
Message-ID: <013a01c8e861$68d6abb0$3a840310$@com>

All,

 

I have a TCP interface between an Erlang system and a C system.  Both
send/receive marshaled binary Erlang terms and I have not had any problems
to date.

 

Today I began doing some more serious testing with larger chunks of binary
to be decoded in C.

 

We ran into a bug (it seems) with erl_interface 3.5.5.4 that is causing it
to segfault during decoding.  The backtrace looks like this:

 

Program received signal SIGSEGV, Segmentation fault.

[Switching to Thread 46912496233216 (LWP 4091)]

0x00000000004032c4 in _erl_free_term ()

(gdb) bt

#0  0x00000000004032c4 in _erl_free_term ()

#1  0x000000000040496b in erl_decode_it ()

#2  0x0000000000404937 in erl_decode_it ()

#3  0x0000000000404c93 in erl_decode_it ()

#4  0x0000000000404c93 in erl_decode_it ()

#5  0x0000000000405311 in erl_decode ()

#6  0x0000000000401b38 in main (argc=1, argv=0x7fff51ec7938) at badtest.c:28

 

The unfortunate part is that the way this large binary term is generated
cannot be done in any kind of sample code (it's being pulled off an external
database).

 

However, I have created a set of test files in C which recreate the
segfault.  I stored the binary in a flat file (as 'badbinary') and have a
testing program which reads it off disk and attempts to decode it.  To prove
the approach is sane (and that this segfault is related to something strange
about the decoding of this particular binary, not the size or general format
of the binary) there is a 'goodbinary' file and testing program for that.

 

To use the test code:

 

Untar/Ungzip the file.  You may need to edit the Makefile to fix the paths
to your erl_interface library.

 

'make' and then you can:

 

./badtest  (this reads 'badbinary' and attempts to decode, causes segfault)

./goodtest (this reads 'goodbinary' and successfully decodes it)  [nearly
identical code to badtest.c but reads different file w/ different size]

 

Also included is

 

./makegoodbin (a simple program that generates a large ETERM in an identical
format to the badbinary but contains duplicated binary data everywhere) 

 

 

Notes:

 

* The marshaled binary erlang term being sent to C can be successfully
decoded/unmarshaled from within Erlang without a problem

* This is reproducible with many different large erlang terms generated from
our database queries.  'makegoodbin.c' creates a term identical in format to
those causing problems, however it does not have the random distribution of
binary sizes and content, and so I'm not able to reproduce the problem in
this way.

* The entire system, end-to-end including this decoding step, works
perfectly in most cases.  However when the data goes into the 100k+ range,
the segfaults start to happen.  That's why I created the 'makegoodbin' which
follows the same format.  Unfortunately that works even at sizes of >1MB
adding to the confusion of the problem.

 

 

Any help is appreciated.  Thanks.

 

 

Jonathan Gray

Streamy Inc.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080719/d62565b3/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: erl_decode_segfault_test.tar.gz
Type: application/x-gzip
Size: 31030 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20080719/d62565b3/attachment.bin>