From jan@REDACTED Tue Mar 10 23:40:44 2009 From: jan@REDACTED (Jan Lehnardt) Date: Tue, 10 Mar 2009 23:40:44 +0100 Subject: [erlang-patches] Patch for term_to_binary() stack overflow Message-ID: <4918B149-A8CB-472F-A2DE-DEECDF02FB50@apache.org> Dear OTP Team, https://issues.apache.org/jira/browse/COUCHDB-275 describes an issue with Erlang's term_to_binary() BIF and includes a patch. The core of the issue is that term_to_binary() is implemented in a recursive C function that blows the stack when converting deeply nested term structures. We see this when people try to read deeply nested JSON structures that we convert from Erlang terms to binaries representing JSON for delivery. The easiest way to reproduce this is term_to_binary(lists:foldl(fun(E,A) -> [E, A] end, [], lists:seq(1, 100000))). See https://issues.apache.org/jira/secure/attachment/12401866/term_to_binary_fix.diff for a non-recursive version of term_to_binary(). How do other projects deal with this issue? If there is no workaround, is it possible to apply the patch in the next OTP release? On behalf of Damien Katz and the CouchDB team, Jan Lehnardt -- From bernie@REDACTED Wed Mar 11 01:14:01 2009 From: bernie@REDACTED (Bernard Duggan) Date: Wed, 11 Mar 2009 11:14:01 +1100 Subject: [erlang-patches] Spurious(?) dialyzer warnings for built-ins Message-ID: <49B70249.8010202@m5net.com> Hi all, There's been a couple of questions on the -questions list recently about what look to be spurious "Call to missing or unexported function" warnings generated by dialyzer about the use of built-in functions (the specific one that we saw was re:compile/2, but there are others too). The only response I've seen has been "just ignore it" - it seems that if a warning is being generated that should always be ignored, it's probably preferable not to generate it in the first place. Therefore, here's a simple patch against 12B-5 which removes the particular warning - I'm not aware of any issues this should cause, but nor am I a dialyzer expert and I'm happy to be corrected (or indeed to hear a reason why we should generate this warning). Cheers, Bernard -------------- next part -------------- A non-text attachment was scrubbed... Name: dialyzer_builtin.patch Type: text/x-diff Size: 639 bytes Desc: not available URL: From bgustavsson@REDACTED Wed Mar 11 08:15:45 2009 From: bgustavsson@REDACTED (Bjorn Gustavsson) Date: Wed, 11 Mar 2009 08:15:45 +0100 Subject: [erlang-patches] Patch for term_to_binary() stack overflow In-Reply-To: <4918B149-A8CB-472F-A2DE-DEECDF02FB50@apache.org> References: <4918B149-A8CB-472F-A2DE-DEECDF02FB50@apache.org> Message-ID: <6672d0160903110015g61650b1dp2385516e05381c74@mail.gmail.com> On Tue, Mar 10, 2009 at 11:40 PM, Jan Lehnardt wrote: > Dear OTP Team, > > https://issues.apache.org/jira/browse/COUCHDB-275 describes an issue > with Erlang's term_to_binary() BIF and includes a patch. The core of the > issue is that term_to_binary() is implemented in a recursive C function > that blows the stack when converting deeply nested term structures. We > see this when people try to read deeply nested JSON structures that we > convert from Erlang terms to binaries representing JSON for delivery. > > The easiest way to reproduce this is > > term_to_binary(lists:foldl(fun(E,A) -> [E, A] end, [], lists:seq(1, > 100000))). > > See https://issues.apache.org/jira/secure/attachment/12401866/term_to_binary_fix.diff > > for a non-recursive version of term_to_binary(). > > How do other projects deal with this issue? If there is no workaround, > is > it possible to apply the patch in the next OTP release? We will not include the suggested patch in OTP. However, we do intend to fix the problem as soon as possible, in a way that degrades performance as little as possible, possibly already in the R13B release. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From tobias.lindahl@REDACTED Wed Mar 11 11:11:45 2009 From: tobias.lindahl@REDACTED (Tobias Lindahl) Date: Wed, 11 Mar 2009 11:11:45 +0100 Subject: [erlang-patches] Spurious(?) dialyzer warnings for built-ins In-Reply-To: <49B70249.8010202@m5net.com> References: <49B70249.8010202@m5net.com> Message-ID: <49B78E61.8080600@it.uu.se> Hi Bernard, This is already taken care of in the development branch for R13. Although your patch solves the immediately visible problem, it is not the proper way to handle it. Dialyzer needs to have information about all built-in functions for the analysis, and the warning tells us (the developers) that this is not the case. The warning should not have appeared in the released otp, it was a mistake due to lack of communication. Cheers, Tobias PS We do appreciate you sending us the patch and encourage others to do the same. Bernard Duggan wrote: > Hi all, > There's been a couple of questions on the -questions list recently > about what look to be spurious "Call to missing or unexported function" > warnings generated by dialyzer about the use of built-in functions (the > specific one that we saw was re:compile/2, but there are others too). > The only response I've seen has been "just ignore it" - it seems that if > a warning is being generated that should always be ignored, it's > probably preferable not to generate it in the first place. > Therefore, here's a simple patch against 12B-5 which removes the > particular warning - I'm not aware of any issues this should cause, but > nor am I a dialyzer expert and I'm happy to be corrected (or indeed to > hear a reason why we should generate this warning). > > Cheers, > > Bernard > > > > ------------------------------------------------------------------------ > > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://www.erlang.org/mailman/listinfo/erlang-patches From magnus@REDACTED Wed Mar 18 18:04:03 2009 From: magnus@REDACTED (Magnus Henoch) Date: Wed, 18 Mar 2009 17:04:03 +0000 Subject: [erlang-patches] erl -make should set exit code on failure Message-ID: <84iqm6epa4.fsf@linux-b2a3.site> This is just something I need to get off my chest... It always bothered me that "erl -make" would return exit code 0 regardless of whether the compilation succeeded or failed. So here is a patch (against R13A) that sets exit code 1 on failure. I hope it will be useful to someone. --- otp_src_R13A/erts/etc/common/erlexec.c 2009/03/18 16:37:47 1.1 +++ otp_src_R13A/erts/etc/common/erlexec.c 2009/03/18 16:40:23 1.2 @@ -678,7 +678,7 @@ * on itself here. We'll avoid doing that. */ if (strcmp(argv[i], "-make") == 0) { - add_args("-noshell", "-noinput", "-s", "make", "all", NULL); + add_args("-noshell", "-noinput", "-s", "make", "all_or_nothing", NULL); add_Eargs("-B"); haltAfterwards = 1; i = argc; /* Skip rest of command line */ --- otp_src_R13A/lib/tools/src/make.erl 2009/03/18 16:38:24 1.1 +++ otp_src_R13A/lib/tools/src/make.erl 2009/03/18 16:39:46 1.2 @@ -24,12 +24,20 @@ %% If Emakefile is missing the current directory is used. -module(make). --export([all/0,all/1,files/1,files/2]). +-export([all_or_nothing/0,all/0,all/1,files/1,files/2]). -include_lib("kernel/include/file.hrl"). -define(MakeOpts,[noexec,load,netload,noload]). +all_or_nothing() -> + case all() of + up_to_date -> + up_to_date; + error -> + halt(1) + end. + all() -> all([]). -- Magnus Henoch, magnus@REDACTED Erlang Training and Consulting http://www.erlang-consulting.com/ From bgustavsson@REDACTED Fri Mar 20 14:40:35 2009 From: bgustavsson@REDACTED (Bjorn Gustavsson) Date: Fri, 20 Mar 2009 14:40:35 +0100 Subject: [erlang-patches] Patch for term_to_binary() stack overflow In-Reply-To: <4918B149-A8CB-472F-A2DE-DEECDF02FB50@apache.org> References: <4918B149-A8CB-472F-A2DE-DEECDF02FB50@apache.org> Message-ID: <6672d0160903200640l2d427cceqb25a249661ce579a@mail.gmail.com> On Tue, Mar 10, 2009 at 11:40 PM, Jan Lehnardt wrote: > Dear OTP Team, > > https://issues.apache.org/jira/browse/COUCHDB-275 describes an issue > with Erlang's term_to_binary() BIF and includes a patch. The core of the > issue is that term_to_binary() is implemented in a recursive C function > that blows the stack when converting deeply nested term structures. We > see this when people try to read deeply nested JSON structures that we > convert from Erlang terms to binaries representing JSON for delivery. > > The easiest way to reproduce this is > > term_to_binary(lists:foldl(fun(E,A) -> [E, A] end, [], lists:seq(1, > 100000))). > We have now removed deep recursion from the implementation of term_to_binary/1 (will appear in the R13B release). Seems to be somewhat faster than before, at least for deeply nested terms. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From helozjisky.lee@REDACTED Mon Mar 23 15:17:46 2009 From: helozjisky.lee@REDACTED (Jason Lee) Date: Mon, 23 Mar 2009 22:17:46 +0800 Subject: [erlang-patches] (no subject) Message-ID: <1237817866.12236.1.camel@bourne.rtm> From per@REDACTED Mon Mar 30 14:19:20 2009 From: per@REDACTED (Per Hedeland) Date: Mon, 30 Mar 2009 14:19:20 +0200 (CEST) Subject: [erlang-patches] [erlang-questions] IPv6 In-Reply-To: <200903271622.n2RGMPQm071931@pluto.hedeland.org> Message-ID: <200903301219.n2UCJKlZ002391@pluto.hedeland.org> Per Hedeland wrote: > >Is there some good reason for gen_tcp/gen_udp not auto-recognizing an >8-tuple as an IPv6 address? And if there isn't, perhaps the trivial patches below could be considered for inclusion? --Per --- otp_src_R13A/lib/kernel/src/gen_tcp.erl.ORIG 2009-03-12 13:19:00.000000000 +0100 +++ otp_src_R13A/lib/kernel/src/gen_tcp.erl 2009-03-27 17:59:12.000000000 +0100 @@ -46,7 +46,7 @@ end. connect1(Address,Port,Opts,Timer) -> - Mod = mod(Opts), + Mod = mod(Opts ++ [{ip, Address}]), case Mod:getaddrs(Address,Timer) of {ok,IPs} -> case Mod:getserv(Port) of @@ -179,13 +179,17 @@ %% Get the tcp_module mod() -> inet_db:tcp_module(). -%% Get the tcp_module, but option tcp_module|inet|inet6 overrides +%% Get the tcp_module, but option tcp_module|inet|inet6|{ip,tuple()} overrides mod([{tcp_module,Mod}|_]) -> Mod; mod([inet|_]) -> inet_tcp; mod([inet6|_]) -> inet6_tcp; +mod([{ip, {_,_,_,_}}|_]) -> + inet_tcp; +mod([{ip, {_,_,_,_,_,_,_,_}}|_]) -> + inet6_tcp; mod([_|Opts]) -> mod(Opts); mod([]) -> --- otp_src_R13A/lib/kernel/src/gen_udp.erl.ORIG 2009-03-12 13:19:00.000000000 +0100 +++ otp_src_R13A/lib/kernel/src/gen_udp.erl 2009-03-27 18:05:02.000000000 +0100 @@ -104,13 +104,17 @@ %% Get the udp_module mod() -> inet_db:udp_module(). -%% Get the udp_module, but option udp_module|inet|inet6 overrides +%% Get the udp_module, but option tcp_module|inet|inet6|{ip,tuple()} overrides mod([{udp_module,Mod}|_]) -> Mod; mod([inet|_]) -> inet_udp; mod([inet6|_]) -> inet6_udp; +mod([{ip, {_,_,_,_}}|_]) -> + inet_udp; +mod([{ip, {_,_,_,_,_,_,_,_}}|_]) -> + inet6_udp; mod([_|Opts]) -> mod(Opts); mod([]) -> Original message---------------------------------------------------- >From: Per Hedeland >To: erlang-questions@REDACTED >Subject: [erlang-questions] IPv6 > >Hi, > >Is there some good reason for gen_tcp/gen_udp not auto-recognizing an >8-tuple as an IPv6 address? gen_tcp:listen/2 and gen_udp:open/2 don't >even fail if you give them one w/o explicit 'inet6' option, but (OS >stack permitting) interprets "the first 4 bytes" as an IPv4 address - >shocking behaviour for an Erlang module! > >Erlang R13A (erts-5.7) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] >Eshell V5.7 (abort with ^G) >1> gen_tcp:listen(1234, [{ip,{1,43981,4660,0,0,0,0,5}}]). >{ok,#Port<0.439>} >2> gen_udp:open(1234, [{ip,{1,43981,4660,0,0,0,0,5}}]). >{ok,#Port<0.449>} > >$ netstat -an | grep 1234 >tcp 0 0 0.1.171.205:1234 0.0.0.0:* LISTEN >udp 0 0 0.1.171.205:1234 0.0.0.0:* > >'inet' may be a reasonable default (though it isn't actually documented >as such as far as I can see), but blindly "applying" it in the face of >obvious evidence to the contrary can't be right. gen_tcp:connect/3 and >gen_udp:send/4 also have "interesting" behaviour: > >4> gen_tcp:connect({1,43981,4660,0,0,0,0,5}, 1234, []). >{error,nxdomain} >5> gen_udp:send(S, {1,43981,4660,0,0,0,0,5}, 1234, <<"hello">>). >{error,nxdomain} > >Yeah, I can imagine that the name server had trouble finding that >"domain"... (no, it didn't actually try a DNS lookup). > >--Per Hedeland >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://www.erlang.org/mailman/listinfo/erlang-questions >