From fdmanana@REDACTED Thu Sep 1 03:11:13 2011 From: fdmanana@REDACTED (Filipe David Manana) Date: Wed, 31 Aug 2011 18:11:13 -0700 Subject: [erlang-bugs] Bug in enif_compare/2 for bignums In-Reply-To: References: Message-ID: Nasty bug. I've found this apparently can only happen in 64 bits machines, where sizeof(Sint) is smaller than sizeof(int). The following patch fixes the issue for me: https://github.com/fdmanana/otp/commit/a7c3edf11620cf09074da98fd0998f195cb2186e regards, On Wed, Aug 31, 2011 at 11:20 AM, Paul Davis wrote: > Hello, > > I think I've uncovered a bug in enif_compare/2 for bignums. Converting > the same arguments to ErlNifSInt64 values and using C comparison > returns the correct result when enif_compare does not. There's a > reduced test case at [1]. If there's nothing obvious I'll spend some > time later digging into Erlang to see if I can spot the bug. > > > Thanks, > Paul Davis > > [1] https://github.com/davisp/enif_cmp_bug > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > -- Filipe David Manana, fdmanana@REDACTED, fdmanana@REDACTED "Reasonable men adapt themselves to the world. ?Unreasonable men adapt the world to themselves. ?That's why all progress depends on unreasonable men." From pguyot@REDACTED Fri Sep 2 18:17:54 2011 From: pguyot@REDACTED (Paul Guyot) Date: Fri, 2 Sep 2011 18:17:54 +0200 Subject: [erlang-bugs] Pattern matching of floats does not work in the shell Message-ID: <01DB17AC-C0D7-482C-97C4-B255D4F64995@kallisys.net> Hello, We've just noticed that simple pattern matching of 64 bits floats do not work in the shell or in the eval parameter of erl(1) For example, the following line will crash: erl -eval '<> = <<1.4:64/float>>,init:stop().' -noshell Or with the following session: ----- Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.5 (abort with ^G) 1> B = <<64,46,0,0,0,0,0,0>>. <<64,46,0,0,0,0,0,0>> 2> <> = B. ** exception error: no match of right hand side value <<64,46,0,0,0,0,0,0>> ----- While pattern matching with integers works: erl -eval '<> = <<1:64>>,init:stop().' -noshell The pattern matching code works properly when compiled (BEAM and HiPE). Paul -- Semiocast http://semiocast.com/ +33.183627948 - 20 rue Lacaze, 75014 Paris From bgreence@REDACTED Fri Sep 2 21:53:25 2011 From: bgreence@REDACTED (bruce green) Date: Fri, 2 Sep 2011 23:53:25 +0400 Subject: [erlang-bugs] terminate_child possible bug Message-ID: Hi there, The sequence of calls supervisor:terminate_child/2 supervisor:restart_child/2 works in r13b3 and doesn't work in r14b3: R14b3 gives {error,not_found} as if there is no child specification. And yes, there is no child specification when you manually check after the terminate_child call. Is it an erroneous or intentional behaviour? -- BR /Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sat Sep 3 00:32:20 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 02 Sep 2011 23:32:20 +0100 (BST) Subject: [erlang-bugs] Pattern matching of floats does not work in the shell In-Reply-To: <01DB17AC-C0D7-482C-97C4-B255D4F64995@kallisys.net> Message-ID: Are you sure that <<64,46,0,0,0,0,0,0>> is a legal float. If not then trying to "extract" a float from it is bound to fail. Otherwise I have no problems pattern matching 64 bit floats in the shell: 2> B = <<1.4:64/float>>. <<"??ffffff">> 3> <> = B. <<"??ffffff">> 4> F. 1.4 I am running R14B03. Robert ----- Original Message ----- > Hello, > > We've just noticed that simple pattern matching of 64 bits floats do > not work in the shell or in the eval parameter of erl(1) > > For example, the following line will crash: > erl -eval '<> = <<1.4:64/float>>,init:stop().' -noshell > > Or with the following session: > ----- > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.5 (abort with ^G) > 1> B = <<64,46,0,0,0,0,0,0>>. > <<64,46,0,0,0,0,0,0>> > 2> <> = B. > ** exception error: no match of right hand side value > <<64,46,0,0,0,0,0,0>> > ----- > > While pattern matching with integers works: > erl -eval '<> = <<1:64>>,init:stop().' -noshell > > The pattern matching code works properly when compiled (BEAM and > HiPE). > > Paul > -- > Semiocast http://semiocast.com/ > +33.183627948 - 20 rue Lacaze, 75014 Paris > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > From pguyot@REDACTED Sat Sep 3 08:31:47 2011 From: pguyot@REDACTED (Paul Guyot) Date: Sat, 3 Sep 2011 08:31:47 +0200 Subject: [erlang-bugs] Pattern matching of floats does not work in the shell In-Reply-To: References: Message-ID: <09346213-5C4C-4D0D-9CD3-FBA9A194A480@kallisys.net> Le 3 sept. 2011 ? 00:32, Robert Virding a ?crit : > Are you sure that <<64,46,0,0,0,0,0,0>> is a legal float. If not then trying to "extract" a float from it is bound to fail. Otherwise I have no problems pattern matching 64 bit floats in the shell: > > 2> B = <<1.4:64/float>>. > <<"??ffffff">> > 3> <> = B. > <<"??ffffff">> > 4> F. > 1.4 <<64,46,0,0,0,0,0,0>> is 15.0. 1> <<15.0:64/float>>. <<64,46,0,0,0,0,0,0>> I cannot reproduce your example. 2> B = <<1.4:64/float>>. <<"??ffffff">> 3> <> = B. ** exception error: no match of right hand side value <<"??ffffff">> The same issue occurs with an amd64 R14B03 installation. Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> B = <<1.4:64/float>>. <<"??ffffff">> 2> <> = B. ** exception error: no match of right hand side value <<"??ffffff">> Could this be architecture related? What is your architecture? Paul -- Semiocast http://semiocast.com/ +33.183627948 - 20 rue Lacaze, 75014 Paris From robert.virding@REDACTED Sat Sep 3 12:40:35 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 03 Sep 2011 11:40:35 +0100 (BST) Subject: [erlang-bugs] Pattern matching of floats does not work in the shell In-Reply-To: <09346213-5C4C-4D0D-9CD3-FBA9A194A480@kallisys.net> Message-ID: <0681c8e6-9383-4e58-a05e-5a933516aeec@knuth> That is really strange. I am running on MacOSX 10.6.7 Snow Leopard with an erlang compiled straight from the download. And it works for me with 15.0 as well: Erlang R14B03 (erts-5.8.4) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> B = <<15.0:64/float>>. <<64,46,0,0,0,0,0,0>> 2> <> = B. <<64,46,0,0,0,0,0,0>> 3> F. 15.0 4> You sure you are getting all the right standard libraries? Otherwise it could be architecture related. But you mentioned that it worked compiled which would mean that it is NOT architecture related. Weird. Robert ----- Original Message ----- > Le 3 sept. 2011 ? 00:32, Robert Virding a ?crit : > > > Are you sure that <<64,46,0,0,0,0,0,0>> is a legal float. If not > > then trying to "extract" a float from it is bound to fail. > > Otherwise I have no problems pattern matching 64 bit floats in the > > shell: > > > > 2> B = <<1.4:64/float>>. > > <<"??ffffff">> > > 3> <> = B. > > <<"??ffffff">> > > 4> F. > > 1.4 > > <<64,46,0,0,0,0,0,0>> is 15.0. > > 1> <<15.0:64/float>>. > <<64,46,0,0,0,0,0,0>> > > I cannot reproduce your example. > > 2> B = <<1.4:64/float>>. > <<"??ffffff">> > 3> <> = B. > ** exception error: no match of right hand side value <<"??ffffff">> > > The same issue occurs with an amd64 R14B03 installation. > > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:8:8] [rq:8] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.4 (abort with ^G) > 1> B = <<1.4:64/float>>. > <<"??ffffff">> > 2> <> = B. > ** exception error: no match of right hand side value <<"??ffffff">> > > Could this be architecture related? What is your architecture? > > Paul > -- > Semiocast http://semiocast.com/ > +33.183627948 - 20 rue Lacaze, 75014 Paris > > From pguyot@REDACTED Sat Sep 3 20:05:34 2011 From: pguyot@REDACTED (Paul Guyot) Date: Sat, 3 Sep 2011 20:05:34 +0200 Subject: [erlang-bugs] Pattern matching of floats does not work in the shell In-Reply-To: <0681c8e6-9383-4e58-a05e-5a933516aeec@knuth> References: <0681c8e6-9383-4e58-a05e-5a933516aeec@knuth> Message-ID: Le 3 sept. 2011 ? 12:40, Robert Virding a ?crit : > You sure you are getting all the right standard libraries? Otherwise it could be architecture related. But you mentioned that it worked compiled which would mean that it is NOT architecture related. Weird. After further analysis, the bug is in HiPE and occurs in eval_bits.erl (hence the issue in the shell). It can be reproduced with the following function: bin_to_float(B,S) -> <> = B, F. I only tried the following simpler case, which works (except in the shell, where it is interpreted to eval_bits:get_float/3 which is like bin_to_float/2 above): bin_to_float(B) -> <> = B, F. bin_to_float/1 always work. bin_to_float/2 works when the module is compiled without native, but fails when it is compiled +native. The reason why you could not reproduce the bug is that all my installations are configured with --enable-native-libs, and therefore, module eval_bits is compiled +native. You probably did not pass --enable-native-libs on your Mac. The bug is a typo in lib/hipe/icode/hipe_beam_to_icode.erl. I'm submitting a patch. Paul -- Semiocast http://semiocast.com/ +33.183627948 - 20 rue Lacaze, 75014 Paris From robert.virding@REDACTED Sat Sep 3 22:26:17 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 03 Sep 2011 21:26:17 +0100 (BST) Subject: [erlang-bugs] Pattern matching of floats does not work in the shell In-Reply-To: Message-ID: Good to get an explanation of why it works when it does and otherwise not. No I don't use --enable-native-libs, which in this case is good. Robert ----- Original Message ----- > Le 3 sept. 2011 ? 12:40, Robert Virding a ?crit : > > > You sure you are getting all the right standard libraries? > > Otherwise it could be architecture related. But you mentioned that > > it worked compiled which would mean that it is NOT architecture > > related. Weird. > > After further analysis, the bug is in HiPE and occurs in > eval_bits.erl (hence the issue in the shell). > It can be reproduced with the following function: > > bin_to_float(B,S) -> > <> = B, > F. > > I only tried the following simpler case, which works (except in the > shell, where it is interpreted to eval_bits:get_float/3 which is > like bin_to_float/2 above): > > bin_to_float(B) -> > <> = B, > F. > > bin_to_float/1 always work. bin_to_float/2 works when the module is > compiled without native, but fails when it is compiled +native. > > The reason why you could not reproduce the bug is that all my > installations are configured with --enable-native-libs, and > therefore, module eval_bits is compiled +native. You probably did > not pass --enable-native-libs on your Mac. > > The bug is a typo in lib/hipe/icode/hipe_beam_to_icode.erl. I'm > submitting a patch. > > Paul > -- > Semiocast http://semiocast.com/ > +33.183627948 - 20 rue Lacaze, 75014 Paris > > From erlangsiri@REDACTED Mon Sep 5 09:59:10 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Mon, 5 Sep 2011 09:59:10 +0200 Subject: [erlang-bugs] terminate_child possible bug In-Reply-To: References: Message-ID: I Bruce - could it be that you try to restart a temporary child? This was earlier possible to do with restart_child, but that functionality has been removed - from the release note: Supervisors should not save child-specs for temporary processes when they terminate as they should not be restarted. Saving the temporary child spec will result in that you can not start a new temporary process with the same child spec as an already terminated temporary process. Since R14B02 you can not restart a temporary temporary process as arguments are no longer saved, it has however always been semantically incorrect to restart a temporary process. Thanks to Filipe David Manana for reporting this and suggesting a solution. Own Id: OTP-9167 Aux Id: OTP-9064 Supervisors will no longer save start parameters for temporary processes as they will not be restarted. In the case of simple_one_for_one workers such as ssl-connection processes this will substantial reduce the memory footprint of the supervisor. Own Id: OTP-9064 Regards /siri 2011/9/2 bruce green > Hi there, > The sequence of calls > supervisor:terminate_child/2 > supervisor:restart_child/2 > works in r13b3 and doesn't work in r14b3: > > R14b3 gives {error,not_found} as if there is no child specification. > And yes, there is no child specification when you manually check after > the terminate_child call. > > Is it an erroneous or intentional behaviour? > -- > BR /Bruce > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.hughes@REDACTED Mon Sep 5 16:18:22 2011 From: john.hughes@REDACTED (John Hughes) Date: Mon, 5 Sep 2011 16:18:22 +0200 Subject: [erlang-bugs] type improper_list(_,_) undefined Message-ID: <06B6148B41754674AD53BAD860BAC91E@JohnsTablet2009> It looks as though the type improper_list/2 isn't defined in R14B02. Here's my file: -module(bug). -type ok() :: list(float()). -type foo() :: improper_list(float(),integer()). -type baz() :: maybe_improper_list(float(),integer()). Here's the output: 47> c(bug). ./bug.erl:5: type improper_list(_,_) undefined ./bug.erl:4: Warning: type ok() is unused ./bug.erl:5: Warning: type foo() is unused ./bug.erl:6: Warning: type baz() is unused error And here's the documentation: List :: list(Type) %% Proper list ([]-terminated) | improper_list(Type1, Type2) %% Type1=contents, Type2=termination | maybe_improper_list(Type1, Type2) %% Type1 and Type2 as above Either I'm missing something very obvious, or the compiler doesn't know about improper_list. John From kostis@REDACTED Mon Sep 5 17:20:49 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 05 Sep 2011 18:20:49 +0300 Subject: [erlang-bugs] type improper_list(_,_) undefined In-Reply-To: <06B6148B41754674AD53BAD860BAC91E@JohnsTablet2009> References: <06B6148B41754674AD53BAD860BAC91E@JohnsTablet2009> Message-ID: <4E64E8D1.9000605@cs.ntua.gr> On 09/05/11 17:18, John Hughes wrote: > It looks as though the type improper_list/2 isn't defined in R14B02. > Here's my file: > > -module(bug). > > -type ok() :: list(float()). > -type foo() :: improper_list(float(),integer()). > -type baz() :: maybe_improper_list(float(),integer()). > > Here's the output: > > 47> c(bug). > ./bug.erl:5: type improper_list(_,_) undefined > ./bug.erl:4: Warning: type ok() is unused > ./bug.erl:5: Warning: type foo() is unused > ./bug.erl:6: Warning: type baz() is unused > error > > And here's the documentation: > > List :: list(Type) %% Proper list ([]-terminated) > | improper_list(Type1, Type2) %% Type1=contents, Type2=termination > | maybe_improper_list(Type1, Type2) %% Type1 and Type2 as above > > Either I'm missing something very obvious, or the compiler doesn't know > about improper_list. The documentation needs updating. Use nonempty_improper_list/2 instead. It's not only that we really wanted to discourage the use of these types, but as a primitive, improper_list/2 does not make much sense when the list can be empty. If you really need that type you can always define it yourself as: -type improper_list(T1,T2) :: [] | nonempty_improper_list(T1,T2). Kostis From sverker@REDACTED Mon Sep 5 19:45:53 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Mon, 5 Sep 2011 19:45:53 +0200 Subject: [erlang-bugs] a crypto function call crashes VM In-Reply-To: References: Message-ID: <4E650AD1.5090109@erix.ericsson.se> SUZUKI Tetsuya wrote: > Hi, > > BEAM crashes when the following message digest functions using > contexts in crypto module receive some binary as context. > (Environment: Erlang R14B03, Mac OS X 10.6.8 (i386, MacBook Air). > > * md4_update/2 > * md4_final/2 > * md5_update/2 > * md5_final/2 > * sha_update/2 > * sha_final/2 > * hmac_update/2 > * hmac_final/2 > > > : > If the functions communicate with BEAM using resource objects > include MD5_CTX data, BEAM may not crash. > Yes, that would be the nice way to do it. However, changing the existing interface to return resources instead of binaries as context would break backward compatibility. How about three new functions like this: hash_init(Type) -> Context hash_update(Context, Data) -> NewContext hash_final(Context) -> Digest Type = md4 | md5 | sha | hmac Context = NewContext = opaque() [implemented as NIF resource] Data = Digest = binary() And then eventually deprecate the old ones. /Sverker From fdmanana@REDACTED Wed Sep 7 08:28:52 2011 From: fdmanana@REDACTED (Filipe David Manana) Date: Tue, 6 Sep 2011 23:28:52 -0700 Subject: [erlang-bugs] Mac OS X Lion (64bits at least), XCode 4.1 and Erlang OTP R14 Message-ID: I was having an issue with R14 (tried B01, B02, B03 and otp/dev git branch) on Mac OS X Lion with XCode 4.1. Basically very frequent crashes on init:restart/0 calls when the crypto application was loaded. I posted a solution and gdb debug session in case someone finds the same issue: https://gist.github.com/1199903 Hope it helps others. best regards, -- Filipe David Manana, "Reasonable men adapt themselves to the world. ?Unreasonable men adapt the world to themselves. ?That's why all progress depends on unreasonable men." From sverker@REDACTED Wed Sep 7 11:46:50 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 7 Sep 2011 11:46:50 +0200 Subject: [erlang-bugs] Repeatable output from crypto:rand_uniform with negative Low value In-Reply-To: References: Message-ID: <4E673D8A.4000405@erix.ericsson.se> Robert Newson wrote: > Hi, > > I've noticed that repeated calls to crypto:rand_uniform return the > same values if Low is negative and High is (approximately) equal to > -Low, and the answer is also outside of the requested range. > > e.g,; > > 7> crypto:rand_uniform(-10000,10000). > 65536 > 8> crypto:rand_uniform(-10000,10000). > 65536 > 9> crypto:rand_uniform(-10000,10000). > 65536 > > This is with R14B03. > > I've fixed rand_uniform to handle negative number for R14B04. Also made it throw badarg if the second argument 'Hi' is not larger than the first 'Lo'. /Sverker, Erlang/OTP From ferenc.holzhauser@REDACTED Sat Sep 10 00:01:27 2011 From: ferenc.holzhauser@REDACTED (Ferenc Holzhauser) Date: Sat, 10 Sep 2011 00:01:27 +0200 Subject: [erlang-bugs] SSH public key authentication only works on the first key of authorized_keys Message-ID: Hi, Today I stumbled into this old bug (found a post from 2008 about it). Since I need a working SFTP server, I made a simple patch to solve it. It is based on the released R14B03 source. Perhaps it is useful for someone else too. Kind Regards Ferenc -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ssh_file.erl.patch Type: text/x-patch Size: 2659 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ssh_auth.erl.patch Type: text/x-patch Size: 1329 bytes Desc: not available URL: From eric.pailleau@REDACTED Sat Sep 10 22:02:04 2011 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 10 Sep 2011 22:02:04 +0200 Subject: [erlang-bugs] Typo in the web site Message-ID: <4E6BC23C.7030205@wanadoo.fr> Hello, not really important but there is a typo in the dowload page for R14B03 : "The third R14 maintentance release" ^ Cheers. From raimo+erlang-bugs@REDACTED Mon Sep 12 08:50:48 2011 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Mon, 12 Sep 2011 08:50:48 +0200 Subject: [erlang-bugs] Typo in the web site In-Reply-To: <4E6BC23C.7030205@wanadoo.fr> References: <4E6BC23C.7030205@wanadoo.fr> Message-ID: <20110912065048.GA2693@erix.ericsson.se> On Sat, Sep 10, 2011 at 10:02:04PM +0200, PAILLEAU Eric wrote: > Hello, > not really important but there is a typo in the dowload page > for R14B03 : > > "The third R14 maintentance release" > ^ Important or not. Now it is fixed. Thank you for pointing it out. > Cheers. > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From rh@REDACTED Mon Sep 12 16:15:31 2011 From: rh@REDACTED (Robin Haberkorn) Date: Mon, 12 Sep 2011 16:15:31 +0200 (CEST) Subject: [erlang-bugs] different behaviour of re:replace for directly-specified and precompiled regular expressions In-Reply-To: <233b8d0e-a765-4f7b-9abc-36b5319f0bb4@office> Message-ID: <5c231683-fef0-46bf-98c2-b5539243552f@office> Hello, I think I may have discovered a bug in the stdlib 're' module. For some Erlang strings, re:replace behaves differently for regular expressions "re:compile"d with the 'unicode' option and regular expressions passed uncompiled to re:replace, giving 'unicode' in its options list. I've minimized the test case using PropEr. Have a look at the following erl session: Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> RegExp = ".". "." 2> {ok, RegExpC} = re:compile(RegExp, [unicode]). {ok,{re_pattern,0,1, <<69,82,67,80,56,0,0,0,0,8,0,0,0,0,0,0,0,0,0,0,0,0,0,...>>}} 3> re:replace([133], RegExp, " ", [unicode, global]). [<<" ">>] 4> re:replace([133], RegExpC, " ", [global]). [133] 5> unicode:characters_to_binary(re:replace([133], RegExp, " ", [unicode, global])). <<" ">> 6> unicode:characters_to_binary(re:replace([133], RegExpC, " ", [global])). <<194,133>> 7> That is, in (4) the replacement simply isn't performed. [133] should be a valid unicode charlist and 133 a valid unicode codepoint. I've discovered this by running re:replace on io_lib:format return values. If I'm not totally confused by Erlang's Unicode handling, io_lib:format without the unicode translation modifier returns a (deep) list of byte()s. Since they are integer lists the UTF8 binary encoding does not matter and all integers returned are valid unicode code points (unicode:characters_to_binary does not seem to complain about any list that causes these problems with re:replace). Moreover consider the following difference: 10> re:replace([256], RegExpC, " ", [global]). ** exception error: bad argument in function re:replace/4 called as re:replace([256], {re_pattern,0,1, <<69,82,67,80,56,0,0,0,0,8,0,0,0,0,0,0,0,0,0,0,0,0,0,...>>}, " ", [global]) 11> re:replace([256], RegExp, " ", [unicode,global]). [<<" ">>] Almost as if re:replace would expect only byte()s in (10). Is this desired behaviour, perhaps even documented? Best Regards, Robin -- -- ------------------ managed broadband access ------------------ Travelping GmbH phone: +49-391-8190990 Roentgenstr. 13 fax: +49-391-819099299 D-39108 Magdeburg email: info@REDACTED GERMANY web: http://www.travelping.com Company Registration: Amtsgericht Stendal Reg No.: HRB 10578 Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780 -------------------------------------------------------------- From gopienko@REDACTED Wed Sep 14 13:43:07 2011 From: gopienko@REDACTED (Andrew Gopienko) Date: Wed, 14 Sep 2011 18:43:07 +0700 Subject: [erlang-bugs] Check sys.config in release_handler Message-ID: If sys.config in release-upgrade-tarball contains non term data you won't see any errors when will do release upgrade (release_handler:unpack_release, release_handler:install_release, release_handler:make_permanent). But sys.config successfully be copied in releases/VSN and at next restart the node won't be launched. You will get something like that Crash dump was written to: erl_crash.dump could not start kernel pid (application_controller) (error in config file "/.../releases/VSN/sys.config" (1): bad term) Andrew Gopienko Threeline LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Wed Sep 14 15:29:22 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 14 Sep 2011 15:29:22 +0200 Subject: [erlang-bugs] Check sys.config in release_handler In-Reply-To: References: Message-ID: Hi Andrew - Thanks for reporting this. I'll look into it. Regards /siri 2011/9/14 Andrew Gopienko > If sys.config in release-upgrade-tarball contains non term data you won't > see any errors when will do release upgrade > (release_handler:unpack_release, release_handler:install_release, > release_handler:make_permanent). > But sys.config successfully be copied in releases/VSN and at next restart > the node won't be launched. > You will get something like that > > Crash dump was written to: erl_crash.dump > could not start kernel pid (application_controller) (error in config file > "/.../releases/VSN/sys.config" (1): bad term) > > > Andrew Gopienko > Threeline LLC > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mr.z.m.wu@REDACTED Thu Sep 15 05:01:00 2011 From: mr.z.m.wu@REDACTED (zhong ming wu) Date: Wed, 14 Sep 2011 23:01:00 -0400 Subject: [erlang-bugs] make -j probably broken Message-ID: Just to report that compiling the latest stable source is broken when one supplies -j switch to make. I tried make -j3 I would be nice to make use of -j to speed things up on multi-core machines. From bgustavsson@REDACTED Thu Sep 15 07:11:47 2011 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 15 Sep 2011 07:11:47 +0200 Subject: [erlang-bugs] make -j probably broken In-Reply-To: References: Message-ID: On Thu, Sep 15, 2011 at 5:01 AM, zhong ming wu wrote: > Just to report that compiling the latest stable source is broken when > one supplies -j switch to make. ?I tried > > make -j3 > > I would be nice to make use of -j to speed things up on multi-core machines. Using -j for building Erlang/OTP has never worked. We will fix that in the R15 release (which will be released later this year). -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From ess@REDACTED Thu Sep 15 10:36:43 2011 From: ess@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Thu, 15 Sep 2011 10:36:43 +0200 Subject: [erlang-bugs] make -j probably broken In-Reply-To: References: Message-ID: <4E71B91B.5090404@trifork.com> You can try to just re-run it; that usually works for me: make -j3 || make -j3 But of course, it's best by far if it just works; I'm happy to hear that it is planned to be fixed :-) -- "Usually Works" and "Works for Me" are trademarks of GuyOnTheInternet. All rights reserved. zhong ming wu wrote: > Just to report that compiling the latest stable source is broken when > one supplies -j switch to make. I tried > > make -j3 > > I would be nice to make use of -j to speed things up on multi-core machines. > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > From scvalex@REDACTED Thu Sep 15 19:12:15 2011 From: scvalex@REDACTED (Alexandru =?utf-8?B?U2N2b3LFo292?=) Date: Thu, 15 Sep 2011 18:12:15 +0100 Subject: [erlang-bugs] file_server processes very large numbers of messages slowly Message-ID: <20110915171215.GA22278@southbank.eng.vmware.com> Hi, If you try to do a lot of IO concurrently (e.g. 100 000 calls to file:read_file_info/1), you notice that the calls take longer and longer as the number of concurrent requests rises. As far as we (RabbitMQ) can tell, the bottleneck seems to be file_server. For instance, running the attached program (100 000 calls to file:r_f_i) with: erlc breaky.erl && erl +P 1048576 -s breaky break_fileserver -s init stop you can see this happening clearly. The first calls, which happen when file_server is congested with a large number of messages on its queue, take a long time to run; as the messages are processed, the processing speed increases as well (the first column shows the time to serve 1000 requests). Our guess is that this happens because of the selective receive in prim_file:drv_get_response/1. That seems to cause cause a full scan of the mailbox (which is obviously takes longer when there are a lot of messages). Does this sound right? Is there any workaround other than not doing a lot of file IO concurrently? Cheers, Alex -------------- next part -------------- -module(breaky). -compile([export_all]). break_fileserver() -> N = 100000, Self = self(), [spawn(fun () -> {ok, _} = file:read_file_info("breaky.erl"), Self ! done end) || _ <- lists:seq(0,N)], gather(erlang:now(), N). gather(_, 0) -> ok; gather(Now, N) -> Now2 = if N rem 1000 =:= 0 -> Now1 = erlang:now(), io:format("~p: ~p ~p~n", [timer:now_diff(Now1, Now) / 1000, N, process_info(whereis(file_server_2), message_queue_len)]), Now1; true -> Now end, receive done -> gather(Now2, N-1) end. From fritchie@REDACTED Thu Sep 15 20:24:48 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 15 Sep 2011 13:24:48 -0500 Subject: [erlang-bugs] file_server processes very large numbers of messages slowly In-Reply-To: Message of "Thu, 15 Sep 2011 18:12:15 BST." <20110915171215.GA22278@southbank.eng.vmware.com> Message-ID: <7937.1316111088@snookles.snookles.com> Alexandru, I'm about to hop on a plane, so I don't have much time. Yes, your analysis appears sound. There has been discussion in the last week on the mailing list that is related to prim_file and the file_server_2 (registered proc name) process. The probable answer (at least for now) is to write your own interface to the prim_file:read_file_info() (name?) function so that you can take advantage of the "+A n" I/O worker Pthread pool ... assuming of course that your workload doesn't overwhelm them to the point where selective receive doesn't bite you again. -Scott From raimo+erlang-bugs@REDACTED Fri Sep 16 09:15:47 2011 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Fri, 16 Sep 2011 09:15:47 +0200 Subject: [erlang-bugs] file_server processes very large numbers of messages slowly In-Reply-To: <20110915171215.GA22278@southbank.eng.vmware.com> References: <20110915171215.GA22278@southbank.eng.vmware.com> Message-ID: <20110916071547.GA32558@erix.ericsson.se> On Thu, Sep 15, 2011 at 06:12:15PM +0100, Alexandru Scvor?ov wrote: > Hi, > > If you try to do a lot of IO concurrently (e.g. 100 000 calls to > file:read_file_info/1), you notice that the calls take longer and longer > as the number of concurrent requests rises. > > As far as we (RabbitMQ) can tell, the bottleneck seems to be > file_server. > > For instance, running the attached program (100 000 calls to file:r_f_i) with: > erlc breaky.erl && erl +P 1048576 -s breaky break_fileserver -s init stop > you can see this happening clearly. The first calls, which happen when > file_server is congested with a large number of messages on its queue, > take a long time to run; as the messages are processed, the processing > speed increases as well (the first column shows the time to serve 1000 > requests). > > Our guess is that this happens because of the selective receive in > prim_file:drv_get_response/1. That seems to cause cause a full scan of the > mailbox (which is obviously takes longer when there are a lot of messages). > > Does this sound right? Is there any workaround other than not doing a lot > of file IO concurrently? I'd guess you are right, except that you can through 'raw' files do more _file IO_ concurrently. But file:read_file_info/1 does not have any 'raw' possibility. The 'raw' vs non-raw (cooked?) modes have some history and implications. A node can run on a diskless node using file_server_2 on one other node and by that see the other nodes filesystem. So the whole file server subsystem is geared around this concept. The 'raw' files will see the actual filesystem of the own node, so they can be different filesystems. In most e.g Unix networks the interesting filesystems of one machine look the same from all machines making 'raw' files look like just an speed optimization instead of a different concept. So, you can open a file in 'raw' mode as well as read and write from it, without bothering a central bottleneck, but the meta operations does not (yet) have a 'raw' option. The file you open with a 'raw' option could be a different one than the one you just did read_file_info on if you are running as a fileserver slave against another node's file_server_2. Therefore what is lacking is read_file_info in a 'raw' variant. As well as advise/4, change_group/2, change_owner/2,3, change_time/2,3 ... ... write_file/2,3, write_file_info/2. A total of around 28 operations. A strategy is needed, either for naming, adding an option list to all calls, or a file:raw(Operation, ...) ... / Raimo > > Cheers, > Alex > > -module(breaky). > -compile([export_all]). > > break_fileserver() -> > N = 100000, > Self = self(), > [spawn(fun () -> > {ok, _} = file:read_file_info("breaky.erl"), > Self ! done > end) || _ <- lists:seq(0,N)], > gather(erlang:now(), N). > > gather(_, 0) -> > ok; > gather(Now, N) -> > Now2 = if N rem 1000 =:= 0 -> > Now1 = erlang:now(), > io:format("~p: ~p ~p~n", [timer:now_diff(Now1, Now) / 1000, > N, > process_info(whereis(file_server_2), > message_queue_len)]), > Now1; > true -> Now > end, > receive > done -> gather(Now2, N-1) > end. > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From apage43@REDACTED Sat Sep 17 06:43:14 2011 From: apage43@REDACTED (Aaron Miller) Date: Fri, 16 Sep 2011 21:43:14 -0700 Subject: [erlang-bugs] Bug in .ZIP file reader Message-ID: <1B39511C33E3472793A1B7E4B8E0F04D@ninjawhale.com> The .ZIP data descriptor block, if present is stored after the file data in a .ZIP file, but prim_zip handles it as if it were -before- the file data. http://en.wikipedia.org/wiki/ZIP_(file_format)#File_headers https://github.com/erlang/otp/blob/dev/erts/preloaded/src/prim_zip.erl#L183 A file in a .ZIP with a data descriptor block if compressed will simply fail to read (due to the first 12 bytes being chopped off), and if uncompressed will have their first 12 bytes chopped off and returned with the data descriptor block on the end. I haven't found any ZIP implementations that store uncompressed files with a data descriptor block, but I have created one in the test zip file I have attached. 1> erl_prim_loader:get_file("test.ez/file.compressed"). error 2> erl_prim_loader:get_file("test.ez/file.uncompressed"). {ok,<<"Uncompressed text.\n">>,"test.ez/file.uncompressed"} 3> erl_prim_loader:get_file("test.ez/file.uncompressed.with.descriptor"). {ok,<<32,116,101,120,116,46,10,80,75,7,8,28,191,106,205, 19,0,0,0,19,0,0,0>>, "test.ez/file.uncompressed.with.descriptor"} It works for my purposes to just delete the descriptor handling as I've done here: https://github.com/couchbaselabs/otp/commit/126a990d75311a2da59b6c6e85d05e47edb39a5a#diff-8 But, this will still return data with junk on the end (16 bytes of it, actually, as the descriptor block is 16 bytes long, not 12.), in the case that an uncompressed file is stored with a descriptor block (which doesn't happen in any zip files I have lying around). -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test.ez Type: application/octet-stream Size: 533 bytes Desc: not available URL: From scvalex@REDACTED Tue Sep 27 14:23:03 2011 From: scvalex@REDACTED (Alexandru =?utf-8?B?U2N2b3LFo292?=) Date: Tue, 27 Sep 2011 13:23:03 +0100 Subject: [erlang-bugs] file_server processes very large numbers of messages slowly In-Reply-To: <7937.1316111088@snookles.snookles.com> References: <20110915171215.GA22278@southbank.eng.vmware.com> <7937.1316111088@snookles.snookles.com> Message-ID: <20110927122301.GE3913@southbank.eng.vmware.com> Hi Scott, Thanks a lot for the suggestion. We needed a few more functions than just read_file_info(), including a few from filelib (which used file internally). We ended up writing a few wrappers around the prim_file functions. We also had a bit of problem hitting the open files limit, but we already had a system in place to manage that. Thanks again, Alex On Thu, Sep 15, 2011 at 01:24:48PM -0500, Scott Lystig Fritchie wrote: > Alexandru, I'm about to hop on a plane, so I don't have much time. Yes, > your analysis appears sound. There has been discussion in the last week > on the mailing list that is related to prim_file and the file_server_2 > (registered proc name) process. > > The probable answer (at least for now) is to write your own interface to > the prim_file:read_file_info() (name?) function so that you can take > advantage of the "+A n" I/O worker Pthread pool ... assuming of course > that your workload doesn't overwhelm them to the point where selective > receive doesn't bite you again. > > -Scott > > From wallentin.dahlberg@REDACTED Wed Sep 28 23:14:08 2011 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Wed, 28 Sep 2011 23:14:08 +0200 Subject: [erlang-bugs] file_server processes very large numbers of messages slowly In-Reply-To: <20110927122301.GE3913@southbank.eng.vmware.com> References: <20110915171215.GA22278@southbank.eng.vmware.com> <7937.1316111088@snookles.snookles.com> <20110927122301.GE3913@southbank.eng.vmware.com> Message-ID: Much of the "cooked" file-handling stems from distributed erlang. (Which I now read Raimo explained so on to the next issue ...) Transparent distributed access to a file-system or a fast local file-system? Though a very nifty feature, one can certainly question the reasons for having a file-server being the default behavior in erlang. I would argue against this. A (normal?) three tier solution would probably be better. Tier 1: Gritty driver/nif interface to files. (prim_file) unstable api Tier 2: local file acess (file) stable api Tier 3: distributed access (not file) At first glance Tier 1 and 2 could be collapsed to the same tier but that would not be ideal. I want the prim_* api to be changeable. (Which is the reason why it is not documented I believe) For example, changing the #file_info file times to epochs instead of having *localtime* datetime tuples while the file module still can have its datetime tuples. (This is a change which probably will be in r15). It is unlikely we would remove file-server from file and into a another abstraction layer, but the default behavior should be changed. And file should be extended with the functionality Raimo described. We should probably remove as much functionality as possible in the efile_driver and let higher tier (in erlang) take care of that. Now if I only could find a time-machine or a pill that lets me work 36h a day. // Bj?rn-Egil -- Concern for the past is hindering our future. 2011/9/27 Alexandru Scvor?ov > Hi Scott, > > Thanks a lot for the suggestion. We needed a few more functions than > just read_file_info(), including a few from filelib (which used file > internally). We ended up writing a few wrappers around the prim_file > functions. We also had a bit of problem hitting the open files limit, > but we already had a system in place to manage that. > > Thanks again, > > Alex > > On Thu, Sep 15, 2011 at 01:24:48PM -0500, Scott Lystig Fritchie wrote: > > Alexandru, I'm about to hop on a plane, so I don't have much time. Yes, > > your analysis appears sound. There has been discussion in the last week > > on the mailing list that is related to prim_file and the file_server_2 > > (registered proc name) process. > > > > The probable answer (at least for now) is to write your own interface to > > the prim_file:read_file_info() (name?) function so that you can take > > advantage of the "+A n" I/O worker Pthread pool ... assuming of course > > that your workload doesn't overwhelm them to the point where selective > > receive doesn't bite you again. > > > > -Scott > > > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > -------------- next part -------------- An HTML attachment was scrubbed... URL: