From steven.charles.davis@REDACTED Tue Jan 5 02:22:57 2010 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 04 Jan 2010 19:22:57 -0600 Subject: ERL_LIBS and Windows (minor quirk) Message-ID: <4B429471.3020506@gmail.com> On Windows, and unlike code:add_path, setting the ERL_LIBS environment variable adds paths with a double-backslash... e.g.: ERL_LIBS=D:\Erlang\lib;... ...results in: "D:\\Erlang\\lib/yaws/ebin";... Thus it is possible to add duplicate paths: 1> code:get_path(). [".","D:\\Erlang\\lib/yaws/ebin",...] 2> code:add_patha("D:/Erlang/lib/yaws/ebin"). true 3> code:get_path(). ["d:/Erlang/lib/yaws",".","D:\\Erlang\\lib/yaws/ebin",...] Since code:add_path converts "\\" to "/" you can even do: 2> code:add_patha("D:\\Erlang\\lib/yaws/ebin"). true 3> code:get_path(). ["d:/Erlang/lib/yaws",".","D:\\Erlang\\lib/yaws/ebin",...] ...and get the exact same result! Of course, the real problem here is the design of the underlying OS (in many more ways than just this oddity), but perhaps somebody will be inspired to tidy this one up as it may have some (if rather tiny) effect on the performance of the code server. /s From alex.borovsky@REDACTED Sun Jan 10 18:12:08 2010 From: alex.borovsky@REDACTED (Alexander Borovsky) Date: Sun, 10 Jan 2010 19:12:08 +0200 Subject: float parsing bug Message-ID: <92468cb61001100912s3a465e7drd53d6995cd739b5f@mail.gmail.com> Release: R13B03 Steps to reproduce: 1) erlang:list_to_float("42") Expected result: 42.0 (float) Actual result: error:badarg -- With best regards, Alexander Borovsky From ulf.wiger@REDACTED Sun Jan 10 18:58:42 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 10 Jan 2010 18:58:42 +0100 Subject: [erlang-bugs] float parsing bug In-Reply-To: <92468cb61001100912s3a465e7drd53d6995cd739b5f@mail.gmail.com> References: <92468cb61001100912s3a465e7drd53d6995cd739b5f@mail.gmail.com> Message-ID: <4B4A1552.50604@erlang-consulting.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Alexander Borovsky wrote: > Release: R13B03 > > Steps to reproduce: > 1) erlang:list_to_float("42") > > Expected result: > 42.0 (float) > > Actual result: > error:badarg Arguably, this is in line with the documented behaviour: "list_to_float(String) -> float() Types: String = string() Returns the float whose text representation is String. > list_to_float("2.2017764e+0"). 2.2017764 Failure: badarg if String contains a bad representation of a float." There is no float that has the text representation "42". Eshell V5.7.2 (abort with ^G) 1> is_float(42). false BR, Ulf W -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAktKFVIACgkQtqqFieqzed2K+wCeN/rCVoLwk8XFBIxffc/RZHD8 oPIAn2O+NCdfBrGzkB9kJEHVTmX//uKk =7LZ7 -----END PGP SIGNATURE----- --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From alex.borovsky@REDACTED Sun Jan 10 19:53:40 2010 From: alex.borovsky@REDACTED (Alexander Borovsky) Date: Sun, 10 Jan 2010 20:53:40 +0200 Subject: [erlang-bugs] float parsing bug In-Reply-To: <4B4A1552.50604@erlang-consulting.com> References: <92468cb61001100912s3a465e7drd53d6995cd739b5f@mail.gmail.com> <4B4A1552.50604@erlang-consulting.com> Message-ID: <92468cb61001101053t34b07356r7947e6f536eb09a@mail.gmail.com> On Sun, Jan 10, 2010 at 19:58, Ulf Wiger wrote: > Alexander Borovsky wrote: >> Release: R13B03 >> >> Steps to reproduce: >> 1) erlang:list_to_float("42") >> >> Expected result: >> 42.0 (float) >> >> Actual result: >> error:badarg > > Arguably, this is in line with the documented behaviour: > > "list_to_float(String) -> float() > > Types: > String = string() > > Returns the float whose text representation is String. > >> list_to_float("2.2017764e+0"). > 2.2017764 > > Failure: badarg if String contains a bad representation of a float." > > > There is no float that has the text representation "42". 42.0 could have text representation "42". In most languages (C, Java, Ruby, etc) "42" parsed to float with no errors. Also in some cases float values could be outputted with no decimal point: In C: #include int main() { double d = 42; printf("%g\n", d); return 0; } will print "42" So integers should be properly parsed by list_to_float() function (as corresponding float value) -- With best regards, Alexander Borovsky From fredrik.svahn@REDACTED Sun Jan 10 20:29:04 2010 From: fredrik.svahn@REDACTED (Fredrik Svahn) Date: Sun, 10 Jan 2010 20:29:04 +0100 Subject: [erlang-bugs] float parsing bug In-Reply-To: <92468cb61001101053t34b07356r7947e6f536eb09a@mail.gmail.com> References: <92468cb61001100912s3a465e7drd53d6995cd739b5f@mail.gmail.com> <4B4A1552.50604@erlang-consulting.com> <92468cb61001101053t34b07356r7947e6f536eb09a@mail.gmail.com> Message-ID: See: http://old.nabble.com/list_to_float%28%29-td19309221.html BR /Fredrik On Sun, Jan 10, 2010 at 7:53 PM, Alexander Borovsky wrote: > On Sun, Jan 10, 2010 at 19:58, Ulf Wiger wrote: > >> Alexander Borovsky wrote: >>> Release: R13B03 >>> >>> Steps to reproduce: >>> 1) erlang:list_to_float("42") >>> >>> Expected result: >>> 42.0 (float) >>> >>> Actual result: >>> error:badarg >> >> Arguably, this is in line with the documented behaviour: >> >> "list_to_float(String) -> float() >> >> Types: >> String = string() >> >> Returns the float whose text representation is String. >> >>> list_to_float("2.2017764e+0"). >> 2.2017764 >> >> Failure: badarg if String contains a bad representation of a float." >> >> >> There is no float that has the text representation "42". > > 42.0 could have text representation "42". > > In most languages (C, Java, Ruby, etc) "42" parsed to float with no errors. > > Also in some cases float values could be outputted with no decimal point: > > In C: > #include > > int main() { > ?double d = 42; > ?printf("%g\n", d); > ?return 0; > } > > will print "42" > > So integers should be properly parsed by list_to_float() function (as > corresponding float value) > > -- > With best regards, > Alexander Borovsky > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org > > From ulf.wiger@REDACTED Sun Jan 10 20:31:22 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 10 Jan 2010 20:31:22 +0100 Subject: [erlang-bugs] float parsing bug In-Reply-To: <92468cb61001101053t34b07356r7947e6f536eb09a@mail.gmail.com> References: <92468cb61001100912s3a465e7drd53d6995cd739b5f@mail.gmail.com> <4B4A1552.50604@erlang-consulting.com> <92468cb61001101053t34b07356r7947e6f536eb09a@mail.gmail.com> Message-ID: <4B4A2B0A.70605@erlang-consulting.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Alexander Borovsky wrote: > > 42.0 could have text representation "42". > > In most languages (C, Java, Ruby, etc) "42" parsed to float with no errors. > > Also in some cases float values could be outputted with no decimal point: This is why I wrote "arguably". :) The question has been up on the mailing list before: http://www.erlang.org/cgi-bin/ezmlm-cgi?4:msp:15372:bkelgefglpkakhbedlbf (May 12 2005) http://www.erlang.org/cgi-bin/ezmlm-cgi/4/27647 (Jul 3 2007) http://www.erlang.org/cgi-bin/ezmlm-cgi/4/37899 (Sep 4 2008) In the last one, Richard Carlsson wrote: It's a "feature": list_to_float/1 is a rather low level function, which assumes that you have already checked that the string has the proper form for an Erlang float, i.e., "0.0" and "1.0E-3" are ok, but "22", ".0", and "0." are not. But you're definitely not the first one to disagree with this design decision. BR, UlfW -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAktKKwoACgkQtqqFieqzed3PSACfVOYTeSt9isb25zYygZg9RCXw hGYAoKfAiYDMXnaljZh8gbPDmRVBZM1C =0fkq -----END PGP SIGNATURE----- --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From bgustavsson@REDACTED Mon Jan 11 16:44:09 2010 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 11 Jan 2010 16:44:09 +0100 Subject: [erlang-bugs] open_port cd PortSetting not working on windows In-Reply-To: <523869a70912271546j60b19704we87891d6ceedaee6@mail.gmail.com> References: <523869a70912271546j60b19704we87891d6ceedaee6@mail.gmail.com> Message-ID: <6672d0161001110744sdb4fb63sc6639445fc2a7f34@mail.gmail.com> 2009/12/28 Davide Marqu?s : > Hi all, > > Just found out that the open_port {cd, Dir} PortSetting isn't working on > windows. Which version of Windows? -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From rvirding@REDACTED Mon Jan 11 19:34:51 2010 From: rvirding@REDACTED (Robert Virding) Date: Mon, 11 Jan 2010 19:34:51 +0100 Subject: [erlang-bugs] float parsing bug In-Reply-To: <4B4A2B0A.70605@erlang-consulting.com> References: <92468cb61001100912s3a465e7drd53d6995cd739b5f@mail.gmail.com> <4B4A1552.50604@erlang-consulting.com> <92468cb61001101053t34b07356r7947e6f536eb09a@mail.gmail.com> <4B4A2B0A.70605@erlang-consulting.com> Message-ID: <3dbc6d1c1001111034j1bdf3392gdeb4122386165cf7@mail.gmail.com> 2010/1/10 Ulf Wiger > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Alexander Borovsky wrote: > > > > 42.0 could have text representation "42". > > > > In most languages (C, Java, Ruby, etc) "42" parsed to float with no > errors. > > > > Also in some cases float values could be outputted with no decimal point: > > This is why I wrote "arguably". :) > > The question has been up on the mailing list before: > > http://www.erlang.org/cgi-bin/ezmlm-cgi?4:msp:15372:bkelgefglpkakhbedlbf > (May 12 2005) > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/27647 (Jul 3 2007) > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/37899 (Sep 4 2008) > > In the last one, Richard Carlsson wrote: > > It's a "feature": list_to_float/1 is a rather low level function, > which assumes that you have already checked that the string has the > proper form for an Erlang float, i.e., "0.0" and "1.0E-3" are ok, > but "22", ".0", and "0." are not. > > But you're definitely not the first one to disagree with this design > decision. > I *knew* it was a mistake to add floats in the first place. If NASA can hit the moon without them why should we need them for a telephone exchange or serving up a few pages on the web? Robert From ulf.wiger@REDACTED Wed Jan 13 00:04:24 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 13 Jan 2010 00:04:24 +0100 Subject: erl_scan counts white space badly Message-ID: <4B4CFFF8.5080107@erlang-consulting.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I tried the column tracking functionality of erl_scan: 8> {ok,Fd} = file:open("parse_trans.erl",[read]). {ok,<0.48.0>} 9> {ok,Epp} = epp:open(parse_trans,Fd,{1,1},["."],[]). {ok,<0.50.0>} 10> epp:parse_erl_form(Epp). {ok,{attribute,{1,1},file,{"parse_trans",{1,1}}}} ... 18> epp:parse_erl_form(Epp). {ok,{attribute,{78,2}, record, {context,[{record_field,{78,19},{atom,{78,19},module}}, {record_field,{79,5},{atom,{79,5},function}}, {record_field,{80,5},{atom,{80,5},arity}}, {record_field,{81,19},{atom,{81,19},file}}, {record_field,{82,19},{atom,{82,19},options}}]}}} The original source code wasn't misaligned, but the indentation used a mix of tabs and spaces. erl_scan treats all kinds of white space as one space. I think it would be reasonable to at the very least increment by 8 if it's a tab. Supporting a tab length option might also be possible, but then it would also be nice to have e.g. epp support passing of this option to the scanner. Apologies for not just making my own patch and committing, but I was just playing with this out of curiosity. BR, Ulf W -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAktM//gACgkQtqqFieqzed0z8ACgmrRZHiHMh98aK04JbYFcQuT+ 6AgAnjeee8wG0wUJTTEHA2hhCdQ6GsAP =Dhk8 -----END PGP SIGNATURE----- --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From baggett.patrick@REDACTED Wed Jan 13 02:18:05 2010 From: baggett.patrick@REDACTED (Patrick Baggett) Date: Tue, 12 Jan 2010 19:18:05 -0600 Subject: [erlang-bugs] float parsing bug In-Reply-To: <3dbc6d1c1001111034j1bdf3392gdeb4122386165cf7@mail.gmail.com> References: <92468cb61001100912s3a465e7drd53d6995cd739b5f@mail.gmail.com> <4B4A1552.50604@erlang-consulting.com> <92468cb61001101053t34b07356r7947e6f536eb09a@mail.gmail.com> <4B4A2B0A.70605@erlang-consulting.com> <3dbc6d1c1001111034j1bdf3392gdeb4122386165cf7@mail.gmail.com> Message-ID: <52f46b6b1001121718n73e5478fl866b201c7841b2e8@mail.gmail.com> Wings3D? On Mon, Jan 11, 2010 at 12:34 PM, Robert Virding wrote: > 2010/1/10 Ulf Wiger > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Alexander Borovsky wrote: > > > > > > 42.0 could have text representation "42". > > > > > > In most languages (C, Java, Ruby, etc) "42" parsed to float with no > > errors. > > > > > > Also in some cases float values could be outputted with no decimal > point: > > > > This is why I wrote "arguably". :) > > > > The question has been up on the mailing list before: > > > > http://www.erlang.org/cgi-bin/ezmlm-cgi?4:msp:15372:bkelgefglpkakhbedlbf > > (May 12 2005) > > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/27647 (Jul 3 2007) > > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/37899 (Sep 4 2008) > > > > In the last one, Richard Carlsson wrote: > > > > It's a "feature": list_to_float/1 is a rather low level function, > > which assumes that you have already checked that the string has the > > proper form for an Erlang float, i.e., "0.0" and "1.0E-3" are ok, > > but "22", ".0", and "0." are not. > > > > But you're definitely not the first one to disagree with this design > > decision. > > > > I *knew* it was a mistake to add floats in the first place. If NASA can hit > the moon without them why should we need them for a telephone exchange or > serving up a few pages on the web? > > Robert > From sebastian_egner@REDACTED Wed Jan 13 11:45:29 2010 From: sebastian_egner@REDACTED (Sebastian Egner) Date: Wed, 13 Jan 2010 02:45:29 -0800 (PST) Subject: float parsing bug Message-ID: <885724.67645.qm@web32206.mail.mud.yahoo.com> Hello Alex, Some while ago, I had the same problem: I had to read non-Erlangian floats and solving the problem "at the source", i.e. have the producing program comply with Erlang-syntax, was not really an option. (In my case, the sources are embedded systems that had been rolled out to customers in the hundreds.) My solution is attached to this email: unifloat:to_float/1 parses a deep list of characters into a floating point number, understanding a most liberal floating point syntax. The module is not designed for performance but rather for 'at least you know what you get'. On the design side, separating integers and floats syntactially has its merits in a dynamically typed language like Erlang, but IMHO it would not hurt if the 'result-will-be-float'-parsing functions (list_to_float/1, fread "~g" etc.) would understand a most liberal floating point syntax, making them viable for reading external data sources. Regards, Sebastian. P.S. Did you run into the other "float parsing bug" yet? Erlang floats do not represent NANs, e.g. <> = <<-1:32>> throws a 'no match'. Again, while this is fine within the Erlang world itself, dealing with binary represented floats from external sources can become messy and slow. -------------- next part -------------- A non-text attachment was scrubbed... Name: unifloat.erl Type: application/octet-stream Size: 2874 bytes Desc: not available URL: From rory@REDACTED Thu Jan 14 23:49:40 2010 From: rory@REDACTED (Rory Byrne) Date: Thu, 14 Jan 2010 23:49:40 +0100 Subject: compile/2 ignores unicode flag Message-ID: <20100114224940.GA28093@almeida.jinsky.com> Hi, I ran into problems using re:split with unicode input. The problem appears to be with the re:compile BIF which re:split calls internally. Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> 1> % No problem with re:run 1> re:run("h\x{20AC}llo", "(.)", [unicode, global]). {match,[[{0,1},{0,1}], [{1,3},{1,3}], [{4,1},{4,1}], [{5,1},{5,1}], [{6,1},{6,1}]]} 2> 2> % Problem with compiled version 2> {ok, MP} = re:compile("(.)", [unicode]). {ok,{re_pattern,1,0, <<69,82,67,80,64,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,...>>}} 3> re:run("h\x{20AC}llo", MP, [global]). ** exception error: bad argument in function re:run/3 called as re:run([104,8364,108,108,111], {re_pattern,1,0, <<69,82,67,80,64,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,...>>}, [global]) 4> 4> % Also causes problems for re:split 4> re:split("h\x{20AC}llo", "(.)", [unicode, {return, list}]). [[],"h",[], {incomplete,[],<<"h">>}, [], {error,[],<<"h">>}, [], {error,[],<<"h">>}, [],"l",[],"l",[],"o",[]] 5> Cheers, Rory From cmcknight@REDACTED Wed Jan 13 05:48:44 2010 From: cmcknight@REDACTED (Charles McKnight) Date: Tue, 12 Jan 2010 20:48:44 -0800 Subject: erlang-bugs Digest 12 Jan 2010 23:04:27 -0000 Issue 73 In-Reply-To: <1263337467.26311.ezmlm@erlang.org> References: <1263337467.26311.ezmlm@erlang.org> Message-ID: <100993B1-491C-4682-A528-794BD7534D88@pheonic.com> On Jan 12, 2010, at 3:04 PM, erlang-bugs-digest-help@REDACTED wrote: > > > I *knew* it was a mistake to add floats in the first place. If NASA can hit > the moon without them why should we need them for a telephone exchange or > serving up a few pages on the web? > > Robert Yeah, real men use integers for everything! This float business seems to be the work of some radicals attempting to muck things up! ;-) Glad Ny?ren! Chuck From nesrait@REDACTED Sat Jan 16 17:25:15 2010 From: nesrait@REDACTED (=?ISO-8859-1?Q?Davide_Marqu=EAs?=) Date: Sat, 16 Jan 2010 16:25:15 +0000 Subject: [erlang-bugs] open_port cd PortSetting not working on windows In-Reply-To: <6672d0161001110744sdb4fb63sc6639445fc2a7f34@mail.gmail.com> References: <523869a70912271546j60b19704we87891d6ceedaee6@mail.gmail.com> <6672d0161001110744sdb4fb63sc6639445fc2a7f34@mail.gmail.com> Message-ID: <523869a71001160825y79fc4a8dp7315afcef395ba97@mail.gmail.com> That would be Windows XP. Sorry for not replying sooner. :\ 2010/1/11 Bj?rn Gustavsson > 2009/12/28 Davide Marqu?s : > > Hi all, > > > > Just found out that the open_port {cd, Dir} PortSetting isn't working on > > windows. > > Which version of Windows? > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > From chaoslawful@REDACTED Mon Jan 18 03:57:53 2010 From: chaoslawful@REDACTED (Chaos Wang) Date: Mon, 18 Jan 2010 10:57:53 +0800 Subject: possibly incorrect search order in inet:gethostbyname_tm/4 Message-ID: <4B53CE31.6000904@gmail.com> Hi all, inet:gethostbyname_tm/4 always try any specified DNS resolution methods first, and check whether the given domain name is a IPv4/v6 address when all previous tries failed. So even a string containing valid IP address is specified as domain name to be resolved, it still needs to traverse all the resolution methods before finding out it's already an IP address at last. This would cause serious problems if 'dns' resolution method is specified in some corporation internal networks, in which all unknown domain names (including those treated-as-domain IPv4/v6 address string) will be resolved into the same portal server address. Only 'native' resolution method can be used in such an environment, because libc DNS resolving API will check whether the domain name is an IP address at first. For example, in my working network, the resolving results when specified {lookup,[native]} in kernel inetrc are as following: > inet:getaddr("www.google.com", inet). % real domain name, resolvable at DNS server {ok,{64,233,189,99}} > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP address, not resolvable at DNS server {ok,{10,0,0,2}} But when specified {lookup,[dns]} in kernel inetrc, the results became: > inet:getaddr("www.google.com", inet). % real domain name, resolvable at DNS server {ok,{64,233,189,99}} > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP address, resolved to portal server address by DNS server {ok,{115,124,17,136}} % Oops... IMHO the search order in inet:gethostbyname_tm/4 should be changed to: checking whether the domain name is already a IP address firstly, then try all specified domain resolution methods. Thanks! chaoslawful From hans.bolinder@REDACTED Mon Jan 18 13:37:59 2010 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Mon, 18 Jan 2010 13:37:59 +0100 Subject: [erlang-bugs] erl_scan counts white space badly In-Reply-To: <4B4CFFF8.5080107@erlang-consulting.com> References: <4B4CFFF8.5080107@erlang-consulting.com> Message-ID: <19284.22055.5653.320994@ornendil.du.uab.ericsson.se> [Ulf Wiger:] > I tried the column tracking functionality of erl_scan: ... > The original source code wasn't misaligned, but the indentation > used a mix of tabs and spaces. erl_scan treats all kinds of > white space as one space. > > I think it would be reasonable to at the very least increment by > 8 if it's a tab. Supporting a tab length option might also be > possible, but then it would also be nice to have e.g. epp support > passing of this option to the scanner. We will consider adding such an option in a future release. Best regards, Hans Bolinder, Erlang/OTP team, Ericsson From raimo+erlang-bugs@REDACTED Mon Jan 18 16:07:24 2010 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Mon, 18 Jan 2010 16:07:24 +0100 Subject: [erlang-bugs] possibly incorrect search order in inet:gethostbyname_tm/4 In-Reply-To: <4B53CE31.6000904@gmail.com> References: <4B53CE31.6000904@gmail.com> Message-ID: <20100118150724.GB10908@erix.ericsson.se> On Mon, Jan 18, 2010 at 10:57:53AM +0800, Chaos Wang wrote: > Hi all, > > inet:gethostbyname_tm/4 always try any specified DNS resolution methods > first, and check whether the given domain name is a IPv4/v6 address when > all previous tries failed. So even a string containing valid IP address > is specified as domain name to be resolved, it still needs to traverse > all the resolution methods before finding out it's already an IP address > at last. > > This would cause serious problems if 'dns' resolution method is > specified in some corporation internal networks, in which all unknown > domain names (including those treated-as-domain IPv4/v6 address string) > will be resolved into the same portal server address. Only 'native' > resolution method can be used in such an environment, because libc DNS > resolving API will check whether the domain name is an IP address at first. > > For example, in my working network, the resolving results when specified > {lookup,[native]} in kernel inetrc are as following: > > > inet:getaddr("www.google.com", inet). % real domain name, > resolvable at DNS server > {ok,{64,233,189,99}} > > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP > address, not resolvable at DNS server > {ok,{10,0,0,2}} > > But when specified {lookup,[dns]} in kernel inetrc, the results became: > > > inet:getaddr("www.google.com", inet). % real domain name, > resolvable at DNS server > {ok,{64,233,189,99}} > > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP > address, resolved to portal server address by DNS server > {ok,{115,124,17,136}} % Oops... > > IMHO the search order in inet:gethostbyname_tm/4 should be changed to: > checking whether the domain name is already a IP address firstly, then > try all specified domain resolution methods. > > Thanks! Hi! You make a good case for changing the resolving order. I am almost on your side, there is just one little detail...: Historically, portal server fake IP addresses has not been an issue for inet_res (the DNS resolver). Instead, it has had to balance between the RFCs and what actually is done in product networks. It is not impossible for inet_res to be in an environment where the default domain is foo.bar and a lookup for "17" is supposed to return the IP address for the host 17.foo.bar. Now "17" is not a DNS label according to RFC 1035 section 2.3.1 but that is only a "Preferred name syntax". Today it is more unlikely. But the question still is; when can you safely assume the lookup string at hand is an IP address and not a host name. The existing function inet_parse:ipv4_address is probably too forgiving since it translates "17" -> {0,0,0,17}, "17.18" -> {17,0,0,18}, "17.18.19" -> {17,18,0,19} and "17.18.19.20" -> {17,18,19,20}, all from ancient praxis or even standards. IPv6 addresses are more clear cut since any IPv6 address must contain at least two colons and that is very unlikely for a host name. Can you strengthen your case by finding out more what it takes for libc DNS to be convinced the lookup string is an IPv4 address? > > chaoslawful > > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From paul.joseph.davis@REDACTED Mon Jan 18 22:27:25 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 18 Jan 2010 16:27:25 -0500 Subject: Error in httpd_util:rfc1123_date Message-ID: It would appear that depending on timezone the following call to rfc1123_date breaks with a badarg: 1> httpd_util:rfc1123_date(erlang:localtime()). ** exception error: bad argument in function erlang:universaltime_to_localtime/1 called as erlang:universaltime_to_localtime({{1969,12,31},{23,59,59}}) in call from calendar:local_time_to_universal_time_dst/1 in call from httpd_util:rfc1123_date/1 2> Seeing as that's 1 second before 12:00 AM Jan 1st, 1970 I'm guessing its a miscalculation somewhere. This has been demonstrated in the time zones: Western Indonesian Time And Arabian Standard Time East Africa Time A bit of digging turned up [1] which seems like it might be related. Paul Davis [1] http://www.erlang.org/cgi-bin/ezmlm-cgi/3/334 From raimo+erlang-bugs@REDACTED Tue Jan 19 13:26:28 2010 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Tue, 19 Jan 2010 13:26:28 +0100 Subject: [erlang-bugs] possibly incorrect search order in inet:gethostbyname_tm/4 In-Reply-To: <20100118150724.GB10908@erix.ericsson.se> References: <4B53CE31.6000904@gmail.com> <20100118150724.GB10908@erix.ericsson.se> Message-ID: <20100119122628.GA20697@erix.ericsson.se> I have done some research on my own... These are the ones that succeed (and other numbers within the ranges, of course): Linux, FreeBSD, Solaris: AF_INET "127.0.0.1" -> 127.0.0.1 "192.168.1 -> 192.168.0.1 "10.1" -> 10.0.0.1 "17" -> 0.0.0.17 "192.168.65535" -> 192.168.255.255 "10.16777215" -> 10.255.255.255 "4294967295" -> 255.255.255.255 AF_INET6 "127.0.0.1" -> ::ffff:127.0.0.1 "192.168.1 -> ::ffff:192.168.0.1 "10.1" -> ::ffff:10.0.0.1 "17" -> ::ffff:0.0.0.17 "192.168.65535" -> ::ffff:192.168.255.255 "10.16777215" -> ::ffff:10.255.255.255 "4294967295" -> ::ffff:255.255.255.255 "::127.0.0.1" -> ::127.0.0.1 "::" -> :: FreeBSD (addendum): AF_INET "127.0.0.1." -> 127.0.0.1 OpenBSD: AF_INET "127.0.0.1" -> 127.0.0.1 "127.0.0.1." -> 127.0.0.1 AF_INET6 "::127.0.0.1" -> ::127.0.0.1 "::" -> :: For IPv6 addresses there seems to be consensus: if it parses as an IPv6 address according to the specifications I recall, that IPv6 address is returned, except that OpenBSD does not accept an IPv4 string when requesting an IPv6 address while the others do. For IPv4 addresses Linux, FreeBSD and Solaris regards many numeric strings as IPv4 addresses while OpenBSD requires a 4-field dotted decimal. Both BSDs accept a trailing dot for 4-field dotted decimal, while Linux and Solaris regard a trailing dot as proof that the string is an absolute hostname. Conclusions: The least common denominator (and the most common case) would be to regard 4-field dotted decimal [0..255] with no trailing dot as an IPv4 string. The most widespread behaviour would be the Linux, Solaris and FreeBSD (except for trailing dot) behaviour. And since OpenBSD is not the origin of Erlang/OTP and has little importance in the community, it would probably be the most sensible behaviour. The current inet_parse:ipv4_address/1 needs to be augumented to handle the "192.168.65535", "10.16777215" and "4294967295" IPv4 strings. I'll toss the suggestion around internally and see if and when we can make such a change into the Linux/Solaris behaviour... / Raimo On Mon, Jan 18, 2010 at 04:07:24PM +0100, Raimo Niskanen wrote: > On Mon, Jan 18, 2010 at 10:57:53AM +0800, Chaos Wang wrote: > > Hi all, > > > > inet:gethostbyname_tm/4 always try any specified DNS resolution methods > > first, and check whether the given domain name is a IPv4/v6 address when > > all previous tries failed. So even a string containing valid IP address > > is specified as domain name to be resolved, it still needs to traverse > > all the resolution methods before finding out it's already an IP address > > at last. > > > > This would cause serious problems if 'dns' resolution method is > > specified in some corporation internal networks, in which all unknown > > domain names (including those treated-as-domain IPv4/v6 address string) > > will be resolved into the same portal server address. Only 'native' > > resolution method can be used in such an environment, because libc DNS > > resolving API will check whether the domain name is an IP address at first. > > > > For example, in my working network, the resolving results when specified > > {lookup,[native]} in kernel inetrc are as following: > > > > > inet:getaddr("www.google.com", inet). % real domain name, > > resolvable at DNS server > > {ok,{64,233,189,99}} > > > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP > > address, not resolvable at DNS server > > {ok,{10,0,0,2}} > > > > But when specified {lookup,[dns]} in kernel inetrc, the results became: > > > > > inet:getaddr("www.google.com", inet). % real domain name, > > resolvable at DNS server > > {ok,{64,233,189,99}} > > > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP > > address, resolved to portal server address by DNS server > > {ok,{115,124,17,136}} % Oops... > > > > IMHO the search order in inet:gethostbyname_tm/4 should be changed to: > > checking whether the domain name is already a IP address firstly, then > > try all specified domain resolution methods. > > > > Thanks! > > Hi! > > You make a good case for changing the resolving order. I am almost > on your side, there is just one little detail...: > > Historically, portal server fake IP addresses has not been an issue > for inet_res (the DNS resolver). Instead, it has had to balance between > the RFCs and what actually is done in product networks. > > It is not impossible for inet_res to be in an environment where > the default domain is foo.bar and a lookup for "17" is supposed > to return the IP address for the host 17.foo.bar. Now "17" is > not a DNS label according to RFC 1035 section 2.3.1 but that > is only a "Preferred name syntax". > > Today it is more unlikely. But the question still is; > when can you safely assume the lookup string at hand is > an IP address and not a host name. > > The existing function inet_parse:ipv4_address is probably > too forgiving since it translates "17" -> {0,0,0,17}, > "17.18" -> {17,0,0,18}, "17.18.19" -> {17,18,0,19} > and "17.18.19.20" -> {17,18,19,20}, all from ancient > praxis or even standards. > > IPv6 addresses are more clear cut since any IPv6 address must contain > at least two colons and that is very unlikely for a host name. > > Can you strengthen your case by finding out more what it takes for > libc DNS to be convinced the lookup string is an IPv4 address? > > > > > chaoslawful > > > > > > ________________________________________________________________ > > erlang-bugs mailing list. See http://www.erlang.org/faq.html > > erlang-bugs (at) erlang.org > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From chaoslawful@REDACTED Tue Jan 19 17:22:08 2010 From: chaoslawful@REDACTED (Chaos Wang) Date: Wed, 20 Jan 2010 00:22:08 +0800 Subject: [erlang-bugs] possibly incorrect search order in inet:gethostbyname_tm/4 In-Reply-To: <20100119122628.GA20697@erix.ericsson.se> References: <4B53CE31.6000904@gmail.com> <20100118150724.GB10908@erix.ericsson.se> <20100119122628.GA20697@erix.ericsson.se> Message-ID: <4B55DC30.8080302@gmail.com> Cool~ Sorry for responding so late. I was digging into some glibc source code... The followings are my findings (IPv4 only on Linux). And I totally agree with you, that the safest form to be considered as a IPv4 address will be the standard IPv4 dotted-decimal notation without trailing dot. All libc APIs related to parsing IPv4 address string into in_addr form in my mind are: * inet_addr() (deprecated) * inet_aton() * inet_pton() * gethostbyname() and gethostbyname_r() (obsolete, but used by inet_gethost program) * gethostbyname2() and gethostbyname2_r() (GNU extension) * getaddrinfo() In all these functions, strings with trailing dot will not be considered as IPv4 addresses. inet_aton() (and deprecated inet_addr()) recognize IPv4 numbers-and-dots notation: every dotted number in the address can be in decimal, octal or hexadecimal. And the address can also be written in shorthand: a - means treat a as 32 bits a.b - means treat b as 24 bits a.b.c - means treat c as 16 bits inet_pton() is like inet_aton(), but without all the hexadecimal, octal (with the exception of 0) and shorthand. So it only recognizes standard IPv4 dotted-decimal notation. gethostbyname() (also gethostbyname2() and *_r alternations) use __nss_hostname_digits_dots() to identify IP address. This function calls inet_aton() to parse IPv4 address, except that it refuse to accept any non-digit characters. So the hexadecimal from of IPv4 addresses can't be recognized by it. getaddrinfo() use inet_aton() to recognize IPv4 address. So they are equivalent in IPv4 address parsing. The program I used to test these APIs is in the attachments. Reference locations (in glibc-2.9): * resolv/inet_addr.c implements inet_aton(), inet_addr() * resolv/inet_pton.c implements inet_pton() * sysdeps/posix/getaddrinfo.c implements getaddrinfo() * nss/getXXbyYY.c implements gethostbyname*() * nss/getXXbyYY_r.c implements gethostbyname*_r() * nss/digits_dots.c implements __nss_hostname_digits_dots() Raimo Niskanen wrote: > I have done some research on my own... > > These are the ones that succeed (and other numbers > within the ranges, of course): > > Linux, FreeBSD, Solaris: > AF_INET > "127.0.0.1" -> 127.0.0.1 > "192.168.1 -> 192.168.0.1 > "10.1" -> 10.0.0.1 > "17" -> 0.0.0.17 > "192.168.65535" -> 192.168.255.255 > "10.16777215" -> 10.255.255.255 > "4294967295" -> 255.255.255.255 > AF_INET6 > "127.0.0.1" -> ::ffff:127.0.0.1 > "192.168.1 -> ::ffff:192.168.0.1 > "10.1" -> ::ffff:10.0.0.1 > "17" -> ::ffff:0.0.0.17 > "192.168.65535" -> ::ffff:192.168.255.255 > "10.16777215" -> ::ffff:10.255.255.255 > "4294967295" -> ::ffff:255.255.255.255 > "::127.0.0.1" -> ::127.0.0.1 > "::" -> :: > > FreeBSD (addendum): > AF_INET > "127.0.0.1." -> 127.0.0.1 > > OpenBSD: > AF_INET > "127.0.0.1" -> 127.0.0.1 > "127.0.0.1." -> 127.0.0.1 > AF_INET6 > "::127.0.0.1" -> ::127.0.0.1 > "::" -> :: > > For IPv6 addresses there seems to be consensus: if it > parses as an IPv6 address according to the specifications > I recall, that IPv6 address is returned, except that OpenBSD > does not accept an IPv4 string when requesting an IPv6 > address while the others do. > > For IPv4 addresses Linux, FreeBSD and Solaris regards > many numeric strings as IPv4 addresses while OpenBSD > requires a 4-field dotted decimal. Both BSDs accept > a trailing dot for 4-field dotted decimal, while > Linux and Solaris regard a trailing dot as proof > that the string is an absolute hostname. > > Conclusions: > > The least common denominator (and the most common case) > would be to regard 4-field dotted decimal [0..255] > with no trailing dot as an IPv4 string. > > The most widespread behaviour would be the Linux, Solaris > and FreeBSD (except for trailing dot) behaviour. And since > OpenBSD is not the origin of Erlang/OTP and has little > importance in the community, it would probably be > the most sensible behaviour. > > The current inet_parse:ipv4_address/1 needs to be > augumented to handle the "192.168.65535", > "10.16777215" and "4294967295" IPv4 strings. > > I'll toss the suggestion around internally and see if and when > we can make such a change into the Linux/Solaris behaviour... > > / Raimo > > > > On Mon, Jan 18, 2010 at 04:07:24PM +0100, Raimo Niskanen wrote: > >> On Mon, Jan 18, 2010 at 10:57:53AM +0800, Chaos Wang wrote: >> >>> Hi all, >>> >>> inet:gethostbyname_tm/4 always try any specified DNS resolution methods >>> first, and check whether the given domain name is a IPv4/v6 address when >>> all previous tries failed. So even a string containing valid IP address >>> is specified as domain name to be resolved, it still needs to traverse >>> all the resolution methods before finding out it's already an IP address >>> at last. >>> >>> This would cause serious problems if 'dns' resolution method is >>> specified in some corporation internal networks, in which all unknown >>> domain names (including those treated-as-domain IPv4/v6 address string) >>> will be resolved into the same portal server address. Only 'native' >>> resolution method can be used in such an environment, because libc DNS >>> resolving API will check whether the domain name is an IP address at first. >>> >>> For example, in my working network, the resolving results when specified >>> {lookup,[native]} in kernel inetrc are as following: >>> >>> > inet:getaddr("www.google.com", inet). % real domain name, >>> resolvable at DNS server >>> {ok,{64,233,189,99}} >>> > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP >>> address, not resolvable at DNS server >>> {ok,{10,0,0,2}} >>> >>> But when specified {lookup,[dns]} in kernel inetrc, the results became: >>> >>> > inet:getaddr("www.google.com", inet). % real domain name, >>> resolvable at DNS server >>> {ok,{64,233,189,99}} >>> > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP >>> address, resolved to portal server address by DNS server >>> {ok,{115,124,17,136}} % Oops... >>> >>> IMHO the search order in inet:gethostbyname_tm/4 should be changed to: >>> checking whether the domain name is already a IP address firstly, then >>> try all specified domain resolution methods. >>> >>> Thanks! >>> >> Hi! >> >> You make a good case for changing the resolving order. I am almost >> on your side, there is just one little detail...: >> >> Historically, portal server fake IP addresses has not been an issue >> for inet_res (the DNS resolver). Instead, it has had to balance between >> the RFCs and what actually is done in product networks. >> >> It is not impossible for inet_res to be in an environment where >> the default domain is foo.bar and a lookup for "17" is supposed >> to return the IP address for the host 17.foo.bar. Now "17" is >> not a DNS label according to RFC 1035 section 2.3.1 but that >> is only a "Preferred name syntax". >> >> Today it is more unlikely. But the question still is; >> when can you safely assume the lookup string at hand is >> an IP address and not a host name. >> >> The existing function inet_parse:ipv4_address is probably >> too forgiving since it translates "17" -> {0,0,0,17}, >> "17.18" -> {17,0,0,18}, "17.18.19" -> {17,18,0,19} >> and "17.18.19.20" -> {17,18,19,20}, all from ancient >> praxis or even standards. >> >> IPv6 addresses are more clear cut since any IPv6 address must contain >> at least two colons and that is very unlikely for a host name. >> >> Can you strengthen your case by finding out more what it takes for >> libc DNS to be convinced the lookup string is an IPv4 address? >> >> >>> chaoslawful >>> >>> >>> ________________________________________________________________ >>> erlang-bugs mailing list. See http://www.erlang.org/faq.html >>> erlang-bugs (at) erlang.org >>> >> -- >> >> / Raimo Niskanen, Erlang/OTP, Ericsson AB >> >> ________________________________________________________________ >> erlang-bugs mailing list. See http://www.erlang.org/faq.html >> erlang-bugs (at) erlang.org >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ip.c URL: From kenji.rikitake@REDACTED Wed Jan 20 02:30:31 2010 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Wed, 20 Jan 2010 10:30:31 +0900 Subject: [erlang-bugs] Error in httpd_util:rfc1123_date In-Reply-To: References: Message-ID: <20100120013031.GA9225@k2r.org> Paul Guyot has fixed this, though this hasn't been in the R13B03 yet. http://www.erlang.org/pipermail/erlang-bugs/2008-November/001077.html patch: http://www.erlang.org/pipermail/erlang-bugs/attachments/20081101/c1bab4d9/attachment.obj Kenji Rikitake In the message dated Mon, Jan 18, 2010 at 04:27:01PM -0500, Paul Davis writes: > It would appear that depending on timezone the following call to > rfc1123_date breaks with a badarg: > > 1> httpd_util:rfc1123_date(erlang:localtime()). > ** exception error: bad argument > in function erlang:universaltime_to_localtime/1 > called as erlang:universaltime_to_localtime({{1969,12,31},{23,59,59}}) > in call from calendar:local_time_to_universal_time_dst/1 > in call from httpd_util:rfc1123_date/1 > 2> > > Seeing as that's 1 second before 12:00 AM Jan 1st, 1970 I'm guessing > its a miscalculation somewhere. > > This has been demonstrated in the time zones: > > Western Indonesian Time > And Arabian Standard Time > East Africa Time > > A bit of digging turned up [1] which seems like it might be related. > > > Paul Davis > > > [1] http://www.erlang.org/cgi-bin/ezmlm-cgi/3/334 > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org From raimo+erlang-bugs@REDACTED Wed Jan 20 12:23:22 2010 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Wed, 20 Jan 2010 12:23:22 +0100 Subject: [erlang-bugs] possibly incorrect search order in inet:gethostbyname_tm/4 In-Reply-To: <4B55DC30.8080302@gmail.com> References: <4B53CE31.6000904@gmail.com> <20100118150724.GB10908@erix.ericsson.se> <20100119122628.GA20697@erix.ericsson.se> <4B55DC30.8080302@gmail.com> Message-ID: <20100120112322.GA8363@erix.ericsson.se> On Wed, Jan 20, 2010 at 12:22:08AM +0800, Chaos Wang wrote: > Cool~ > > Sorry for responding so late. I was digging into some glibc source code... > > The followings are my findings (IPv4 only on Linux). And I totally agree > with you, that the safest form to be considered as a IPv4 address will > be the standard IPv4 dotted-decimal notation without trailing dot. But I think the best way would be to adopt the Solaris/Linux behaviour. Perhaps even also the hex/octal notation... > > All libc APIs related to parsing IPv4 address string into in_addr form > in my mind are: > * inet_addr() (deprecated) > * inet_aton() > * inet_pton() > * gethostbyname() and gethostbyname_r() (obsolete, but used by > inet_gethost program) > * gethostbyname2() and gethostbyname2_r() (GNU extension) > * getaddrinfo() > > In all these functions, strings with trailing dot will not be considered > as IPv4 addresses. > > inet_aton() (and deprecated inet_addr()) recognize IPv4 numbers-and-dots > notation: every dotted number in the address can be in decimal, octal or > hexadecimal. And the address can also be written in shorthand: > > a - means treat a as 32 bits > a.b - means treat b as 24 bits > a.b.c - means treat c as 16 bits > > inet_pton() is like inet_aton(), but without all the hexadecimal, octal > (with the exception of 0) and shorthand. So it only recognizes standard > IPv4 dotted-decimal notation. > > gethostbyname() (also gethostbyname2() and *_r alternations) use > __nss_hostname_digits_dots() to identify IP address. This function calls > inet_aton() to parse IPv4 address, except that it refuse to accept any > non-digit characters. So the hexadecimal from of IPv4 addresses can't be > recognized by it. > > getaddrinfo() use inet_aton() to recognize IPv4 address. So they are > equivalent in IPv4 address parsing. I just found out you have almost created the Solaris man page for inet_pton: http://www.s-gms.ms.edus.si/cgi-bin/man-cgi -> search command inet_pton > > The program I used to test these APIs is in the attachments. > > Reference locations (in glibc-2.9): > * resolv/inet_addr.c implements inet_aton(), inet_addr() > * resolv/inet_pton.c implements inet_pton() > * sysdeps/posix/getaddrinfo.c implements getaddrinfo() > * nss/getXXbyYY.c implements gethostbyname*() > * nss/getXXbyYY_r.c implements gethostbyname*_r() > * nss/digits_dots.c implements __nss_hostname_digits_dots() > > Raimo Niskanen wrote: > >I have done some research on my own... > > > >These are the ones that succeed (and other numbers > >within the ranges, of course): > > > >Linux, FreeBSD, Solaris: > > AF_INET > >"127.0.0.1" -> 127.0.0.1 > >"192.168.1 -> 192.168.0.1 > >"10.1" -> 10.0.0.1 > >"17" -> 0.0.0.17 > >"192.168.65535" -> 192.168.255.255 > >"10.16777215" -> 10.255.255.255 > >"4294967295" -> 255.255.255.255 > > AF_INET6 > >"127.0.0.1" -> ::ffff:127.0.0.1 > >"192.168.1 -> ::ffff:192.168.0.1 > >"10.1" -> ::ffff:10.0.0.1 > >"17" -> ::ffff:0.0.0.17 > >"192.168.65535" -> ::ffff:192.168.255.255 > >"10.16777215" -> ::ffff:10.255.255.255 > >"4294967295" -> ::ffff:255.255.255.255 > >"::127.0.0.1" -> ::127.0.0.1 > >"::" -> :: > > > >FreeBSD (addendum): > > AF_INET > >"127.0.0.1." -> 127.0.0.1 > > > >OpenBSD: > > AF_INET > >"127.0.0.1" -> 127.0.0.1 > >"127.0.0.1." -> 127.0.0.1 > > AF_INET6 > >"::127.0.0.1" -> ::127.0.0.1 > >"::" -> :: > > > >For IPv6 addresses there seems to be consensus: if it > >parses as an IPv6 address according to the specifications > >I recall, that IPv6 address is returned, except that OpenBSD > >does not accept an IPv4 string when requesting an IPv6 > >address while the others do. > > > >For IPv4 addresses Linux, FreeBSD and Solaris regards > >many numeric strings as IPv4 addresses while OpenBSD > >requires a 4-field dotted decimal. Both BSDs accept > >a trailing dot for 4-field dotted decimal, while > >Linux and Solaris regard a trailing dot as proof > >that the string is an absolute hostname. > > > >Conclusions: > > > >The least common denominator (and the most common case) > >would be to regard 4-field dotted decimal [0..255] > >with no trailing dot as an IPv4 string. > > > >The most widespread behaviour would be the Linux, Solaris > >and FreeBSD (except for trailing dot) behaviour. And since > >OpenBSD is not the origin of Erlang/OTP and has little > >importance in the community, it would probably be > >the most sensible behaviour. > > > >The current inet_parse:ipv4_address/1 needs to be > >augumented to handle the "192.168.65535", > >"10.16777215" and "4294967295" IPv4 strings. > > > >I'll toss the suggestion around internally and see if and when > >we can make such a change into the Linux/Solaris behaviour... > > > >/ Raimo > > > > > > > >On Mon, Jan 18, 2010 at 04:07:24PM +0100, Raimo Niskanen wrote: > > > >>On Mon, Jan 18, 2010 at 10:57:53AM +0800, Chaos Wang wrote: > >> > >>>Hi all, > >>> > >>>inet:gethostbyname_tm/4 always try any specified DNS resolution methods > >>>first, and check whether the given domain name is a IPv4/v6 address when > >>>all previous tries failed. So even a string containing valid IP address > >>>is specified as domain name to be resolved, it still needs to traverse > >>>all the resolution methods before finding out it's already an IP address > >>>at last. > >>> > >>>This would cause serious problems if 'dns' resolution method is > >>>specified in some corporation internal networks, in which all unknown > >>>domain names (including those treated-as-domain IPv4/v6 address string) > >>>will be resolved into the same portal server address. Only 'native' > >>>resolution method can be used in such an environment, because libc DNS > >>>resolving API will check whether the domain name is an IP address at > >>>first. > >>> > >>>For example, in my working network, the resolving results when specified > >>>{lookup,[native]} in kernel inetrc are as following: > >>> > >>> > inet:getaddr("www.google.com", inet). % real domain name, > >>>resolvable at DNS server > >>> {ok,{64,233,189,99}} > >>> > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP > >>>address, not resolvable at DNS server > >>> {ok,{10,0,0,2}} > >>> > >>>But when specified {lookup,[dns]} in kernel inetrc, the results became: > >>> > >>> > inet:getaddr("www.google.com", inet). % real domain name, > >>>resolvable at DNS server > >>> {ok,{64,233,189,99}} > >>> > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP > >>>address, resolved to portal server address by DNS server > >>> {ok,{115,124,17,136}} % Oops... > >>> > >>>IMHO the search order in inet:gethostbyname_tm/4 should be changed to: > >>>checking whether the domain name is already a IP address firstly, then > >>>try all specified domain resolution methods. > >>> > >>>Thanks! > >>> > >>Hi! > >> > >>You make a good case for changing the resolving order. I am almost > >>on your side, there is just one little detail...: > >> > >>Historically, portal server fake IP addresses has not been an issue > >>for inet_res (the DNS resolver). Instead, it has had to balance between > >>the RFCs and what actually is done in product networks. > >> > >>It is not impossible for inet_res to be in an environment where > >>the default domain is foo.bar and a lookup for "17" is supposed > >>to return the IP address for the host 17.foo.bar. Now "17" is > >>not a DNS label according to RFC 1035 section 2.3.1 but that > >>is only a "Preferred name syntax". > >> > >>Today it is more unlikely. But the question still is; > >>when can you safely assume the lookup string at hand is > >>an IP address and not a host name. > >> > >>The existing function inet_parse:ipv4_address is probably > >>too forgiving since it translates "17" -> {0,0,0,17}, > >>"17.18" -> {17,0,0,18}, "17.18.19" -> {17,18,0,19} > >>and "17.18.19.20" -> {17,18,19,20}, all from ancient > >>praxis or even standards. > >> > >>IPv6 addresses are more clear cut since any IPv6 address must contain > >>at least two colons and that is very unlikely for a host name. > >> > >>Can you strengthen your case by finding out more what it takes for > >>libc DNS to be convinced the lookup string is an IPv4 address? > >> > >> > >>>chaoslawful > >>> > >>> > >>>________________________________________________________________ > >>>erlang-bugs mailing list. See http://www.erlang.org/faq.html > >>>erlang-bugs (at) erlang.org > >>> > >>-- > >> > >>/ Raimo Niskanen, Erlang/OTP, Ericsson AB > >> > >>________________________________________________________________ > >>erlang-bugs mailing list. See http://www.erlang.org/faq.html > >>erlang-bugs (at) erlang.org > >> > > > > > > #include > #include > #include > #include > #include > #include > #include > > void use_inet_addr(const char *name); > void use_inet_aton(const char *name); > void use_inet_pton(const char *name, int af); > void use_gethostbyname(const char *name); > void use_gethostbyname_r(const char *name); > void use_gethostbyname2(const char *name, int af); > void use_gethostbyname2_r(const char *name, int af); > void use_getaddrinfo(const char *name); > > int main(int argc, char *argv[]) > { > if(argc != 2) { > printf("Usage: %s \n", argv[0]); > return 1; > } > > use_inet_addr(argv[1]); > use_inet_aton(argv[1]); > use_inet_pton(argv[1], AF_INET); > use_gethostbyname(argv[1]); > use_gethostbyname_r(argv[1]); > #if defined(_BSD_SOURCE) || defined(_SVID_SOURCE) > use_gethostbyname2(argv[1], AF_INET); > use_gethostbyname2_r(argv[1], AF_INET); > #endif > use_getaddrinfo(argv[1]); > > return 0; > } > > void use_inet_addr(const char *name) > { > in_addr_t addr = inet_addr(name); > > printf("inet_addr: "); > if(addr == INADDR_NONE) { > printf("failed (possibly 255.255.255.255)\n"); > } else { > struct in_addr in; > in.s_addr = addr; > printf("%s\n", inet_ntoa(in)); > } > } > > void use_inet_aton(const char *name) > { > struct in_addr in; > > printf("inet_aton: "); > if(inet_aton(name, &in)) { > printf("%s\n", inet_ntoa(in)); > } else { > printf("failed\n"); > } > } > > void use_inet_pton(const char *name, int af) > { > struct in_addr in; > > printf("inet_pton: "); > if(inet_pton(af, name, &in)) { > printf("%s\n", inet_ntoa(in)); > } else { > printf("failed\n"); > } > } > > void use_gethostbyname(const char *name) > { > struct hostent *h = gethostbyname(name); > > printf("gethostbyname: "); > if(!h) { > printf("failed (%s)\n", hstrerror(h_errno)); > } else { > if(h->h_addrtype != AF_INET) { > printf("failed (invalid address type)\n"); > } else { > char **pp = h->h_addr_list; > while(*pp != NULL) { > struct in_addr *p = (struct in_addr*)(*pp); > printf("%s\n", inet_ntoa(*p)); > ++pp; > } > } > } > } > > void use_gethostbyname_r(const char *name) > { > int rc; > char buf[8192]; > struct hostent h; > struct hostent *rp; > int myerrno; > > printf("gethostbyname_r: "); > rc = gethostbyname_r(name, &h, buf, sizeof(buf), &rp, &myerrno); > if(rc == ERANGE) { > printf("failed (out of memory)\n"); > } else if(!rc) { > if(!rp) { > printf("no address found\n"); > } else { > char **pp = h.h_addr_list; > while(*pp != NULL) { > struct in_addr *p = (struct in_addr*)(*pp); > printf("%s\n", inet_ntoa(*p)); > ++pp; > } > } > } else { > printf("failed (%s)\n", hstrerror(myerrno)); > } > } > > #if defined(_BSD_SOURCE) || defined(_SVID_SOURCE) > > void use_gethostbyname2(const char *name, int af) > { > struct hostent *h = gethostbyname2(name, af); > > printf("gethostbyname2: "); > if(!h) { > printf("failed (%s)\n", hstrerror(h_errno)); > } else { > if(h->h_addrtype != AF_INET) { > printf("failed (invalid address type)\n"); > } else { > char **pp = h->h_addr_list; > while(*pp != NULL) { > struct in_addr *p = (struct in_addr*)(*pp); > printf("%s\n", inet_ntoa(*p)); > ++pp; > } > } > } > } > > void use_gethostbyname2_r(const char *name, int af) > { > int rc; > char buf[8192]; > struct hostent h; > struct hostent *rp; > int myerrno; > > printf("gethostbyname2_r: "); > rc = gethostbyname2_r(name, af, &h, buf, sizeof(buf), &rp, &myerrno); > if(rc == ERANGE) { > printf("failed (out of memory)\n"); > } else if(!rc) { > if(!rp) { > printf("no address found\n"); > } else { > char **pp = h.h_addr_list; > while(*pp != NULL) { > struct in_addr *p = (struct in_addr*)(*pp); > printf("%s\n", inet_ntoa(*p)); > ++pp; > } > } > } else { > printf("failed (%s)\n", hstrerror(myerrno)); > } > } > > #endif > > void use_getaddrinfo(const char *name) > { > int rc; > struct addrinfo hints; > struct addrinfo *resp; > > printf("getaddrinfo: "); > > memset(&hints, 0, sizeof(hints)); > hints.ai_family = AF_INET; > hints.ai_flags = AI_ADDRCONFIG | AI_PASSIVE; > hints.ai_socktype = SOCK_STREAM; > > rc = getaddrinfo(name, NULL, &hints, &resp); > if(rc) { > printf("failed (%s)\n", gai_strerror(rc)); > } else { > struct addrinfo *rp; > > for(rp = resp; rp != NULL; rp = rp->ai_next) { > struct sockaddr_in *addr = (struct sockaddr_in*)(rp->ai_addr); > printf("%s\n", inet_ntoa(addr->sin_addr)); > } > > freeaddrinfo(resp); > } > } > > > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From chaoslawful@REDACTED Wed Jan 20 16:17:37 2010 From: chaoslawful@REDACTED (Chaos Wang) Date: Wed, 20 Jan 2010 23:17:37 +0800 Subject: [erlang-bugs] possibly incorrect search order in inet:gethostbyname_tm/4 In-Reply-To: <20100120112322.GA8363@erix.ericsson.se> References: <4B53CE31.6000904@gmail.com> <20100118150724.GB10908@erix.ericsson.se> <20100119122628.GA20697@erix.ericsson.se> <4B55DC30.8080302@gmail.com> <20100120112322.GA8363@erix.ericsson.se> Message-ID: <4B571E91.204@gmail.com> Raimo Niskanen wrote: > On Wed, Jan 20, 2010 at 12:22:08AM +0800, Chaos Wang wrote: > >> Cool~ >> >> Sorry for responding so late. I was digging into some glibc source code... >> >> The followings are my findings (IPv4 only on Linux). And I totally agree >> with you, that the safest form to be considered as a IPv4 address will >> be the standard IPv4 dotted-decimal notation without trailing dot. >> > > But I think the best way would be to adopt the Solaris/Linux behaviour. > Perhaps even also the hex/octal notation... > Either way is OK to me, as long as it can recognize addresses in standard IPv4 dotted-decimal notation correctly ;-) > >> All libc APIs related to parsing IPv4 address string into in_addr form >> in my mind are: >> * inet_addr() (deprecated) >> * inet_aton() >> * inet_pton() >> * gethostbyname() and gethostbyname_r() (obsolete, but used by >> inet_gethost program) >> * gethostbyname2() and gethostbyname2_r() (GNU extension) >> * getaddrinfo() >> >> In all these functions, strings with trailing dot will not be considered >> as IPv4 addresses. >> >> inet_aton() (and deprecated inet_addr()) recognize IPv4 numbers-and-dots >> notation: every dotted number in the address can be in decimal, octal or >> hexadecimal. And the address can also be written in shorthand: >> >> a - means treat a as 32 bits >> a.b - means treat b as 24 bits >> a.b.c - means treat c as 16 bits >> >> inet_pton() is like inet_aton(), but without all the hexadecimal, octal >> (with the exception of 0) and shorthand. So it only recognizes standard >> IPv4 dotted-decimal notation. >> >> gethostbyname() (also gethostbyname2() and *_r alternations) use >> __nss_hostname_digits_dots() to identify IP address. This function calls >> inet_aton() to parse IPv4 address, except that it refuse to accept any >> non-digit characters. So the hexadecimal from of IPv4 addresses can't be >> recognized by it. >> >> getaddrinfo() use inet_aton() to recognize IPv4 address. So they are >> equivalent in IPv4 address parsing. >> > > I just found out you have almost created the Solaris man page for inet_pton: > http://www.s-gms.ms.edus.si/cgi-bin/man-cgi -> search command inet_pton > Ah, truly a coincidence :-) Manpages in Solaris are more thorough than in Linux, indeed. > >> The program I used to test these APIs is in the attachments. >> >> Reference locations (in glibc-2.9): >> * resolv/inet_addr.c implements inet_aton(), inet_addr() >> * resolv/inet_pton.c implements inet_pton() >> * sysdeps/posix/getaddrinfo.c implements getaddrinfo() >> * nss/getXXbyYY.c implements gethostbyname*() >> * nss/getXXbyYY_r.c implements gethostbyname*_r() >> * nss/digits_dots.c implements __nss_hostname_digits_dots() >> >> Raimo Niskanen wrote: >> >>> I have done some research on my own... >>> >>> These are the ones that succeed (and other numbers >>> within the ranges, of course): >>> >>> Linux, FreeBSD, Solaris: >>> AF_INET >>> "127.0.0.1" -> 127.0.0.1 >>> "192.168.1 -> 192.168.0.1 >>> "10.1" -> 10.0.0.1 >>> "17" -> 0.0.0.17 >>> "192.168.65535" -> 192.168.255.255 >>> "10.16777215" -> 10.255.255.255 >>> "4294967295" -> 255.255.255.255 >>> AF_INET6 >>> "127.0.0.1" -> ::ffff:127.0.0.1 >>> "192.168.1 -> ::ffff:192.168.0.1 >>> "10.1" -> ::ffff:10.0.0.1 >>> "17" -> ::ffff:0.0.0.17 >>> "192.168.65535" -> ::ffff:192.168.255.255 >>> "10.16777215" -> ::ffff:10.255.255.255 >>> "4294967295" -> ::ffff:255.255.255.255 >>> "::127.0.0.1" -> ::127.0.0.1 >>> "::" -> :: >>> >>> FreeBSD (addendum): >>> AF_INET >>> "127.0.0.1." -> 127.0.0.1 >>> >>> OpenBSD: >>> AF_INET >>> "127.0.0.1" -> 127.0.0.1 >>> "127.0.0.1." -> 127.0.0.1 >>> AF_INET6 >>> "::127.0.0.1" -> ::127.0.0.1 >>> "::" -> :: >>> >>> For IPv6 addresses there seems to be consensus: if it >>> parses as an IPv6 address according to the specifications >>> I recall, that IPv6 address is returned, except that OpenBSD >>> does not accept an IPv4 string when requesting an IPv6 >>> address while the others do. >>> >>> For IPv4 addresses Linux, FreeBSD and Solaris regards >>> many numeric strings as IPv4 addresses while OpenBSD >>> requires a 4-field dotted decimal. Both BSDs accept >>> a trailing dot for 4-field dotted decimal, while >>> Linux and Solaris regard a trailing dot as proof >>> that the string is an absolute hostname. >>> >>> Conclusions: >>> >>> The least common denominator (and the most common case) >>> would be to regard 4-field dotted decimal [0..255] >>> with no trailing dot as an IPv4 string. >>> >>> The most widespread behaviour would be the Linux, Solaris >>> and FreeBSD (except for trailing dot) behaviour. And since >>> OpenBSD is not the origin of Erlang/OTP and has little >>> importance in the community, it would probably be >>> the most sensible behaviour. >>> >>> The current inet_parse:ipv4_address/1 needs to be >>> augumented to handle the "192.168.65535", >>> "10.16777215" and "4294967295" IPv4 strings. >>> >>> I'll toss the suggestion around internally and see if and when >>> we can make such a change into the Linux/Solaris behaviour... >>> >>> / Raimo >>> >>> >>> >>> On Mon, Jan 18, 2010 at 04:07:24PM +0100, Raimo Niskanen wrote: >>> >>> >>>> On Mon, Jan 18, 2010 at 10:57:53AM +0800, Chaos Wang wrote: >>>> >>>> >>>>> Hi all, >>>>> >>>>> inet:gethostbyname_tm/4 always try any specified DNS resolution methods >>>>> first, and check whether the given domain name is a IPv4/v6 address when >>>>> all previous tries failed. So even a string containing valid IP address >>>>> is specified as domain name to be resolved, it still needs to traverse >>>>> all the resolution methods before finding out it's already an IP address >>>>> at last. >>>>> >>>>> This would cause serious problems if 'dns' resolution method is >>>>> specified in some corporation internal networks, in which all unknown >>>>> domain names (including those treated-as-domain IPv4/v6 address string) >>>>> will be resolved into the same portal server address. Only 'native' >>>>> resolution method can be used in such an environment, because libc DNS >>>>> resolving API will check whether the domain name is an IP address at >>>>> first. >>>>> >>>>> For example, in my working network, the resolving results when specified >>>>> {lookup,[native]} in kernel inetrc are as following: >>>>> >>>>> > inet:getaddr("www.google.com", inet). % real domain name, >>>>> resolvable at DNS server >>>>> {ok,{64,233,189,99}} >>>>> > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP >>>>> address, not resolvable at DNS server >>>>> {ok,{10,0,0,2}} >>>>> >>>>> But when specified {lookup,[dns]} in kernel inetrc, the results became: >>>>> >>>>> > inet:getaddr("www.google.com", inet). % real domain name, >>>>> resolvable at DNS server >>>>> {ok,{64,233,189,99}} >>>>> > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP >>>>> address, resolved to portal server address by DNS server >>>>> {ok,{115,124,17,136}} % Oops... >>>>> >>>>> IMHO the search order in inet:gethostbyname_tm/4 should be changed to: >>>>> checking whether the domain name is already a IP address firstly, then >>>>> try all specified domain resolution methods. >>>>> >>>>> Thanks! >>>>> >>>>> >>>> Hi! >>>> >>>> You make a good case for changing the resolving order. I am almost >>>> on your side, there is just one little detail...: >>>> >>>> Historically, portal server fake IP addresses has not been an issue >>>> for inet_res (the DNS resolver). Instead, it has had to balance between >>>> the RFCs and what actually is done in product networks. >>>> >>>> It is not impossible for inet_res to be in an environment where >>>> the default domain is foo.bar and a lookup for "17" is supposed >>>> to return the IP address for the host 17.foo.bar. Now "17" is >>>> not a DNS label according to RFC 1035 section 2.3.1 but that >>>> is only a "Preferred name syntax". >>>> >>>> Today it is more unlikely. But the question still is; >>>> when can you safely assume the lookup string at hand is >>>> an IP address and not a host name. >>>> >>>> The existing function inet_parse:ipv4_address is probably >>>> too forgiving since it translates "17" -> {0,0,0,17}, >>>> "17.18" -> {17,0,0,18}, "17.18.19" -> {17,18,0,19} >>>> and "17.18.19.20" -> {17,18,19,20}, all from ancient >>>> praxis or even standards. >>>> >>>> IPv6 addresses are more clear cut since any IPv6 address must contain >>>> at least two colons and that is very unlikely for a host name. >>>> >>>> Can you strengthen your case by finding out more what it takes for >>>> libc DNS to be convinced the lookup string is an IPv4 address? >>>> >>>> >>>> >>>>> chaoslawful >>>>> >>>>> >>>>> ________________________________________________________________ >>>>> erlang-bugs mailing list. See http://www.erlang.org/faq.html >>>>> erlang-bugs (at) erlang.org >>>>> >>>>> >>>> -- >>>> >>>> / Raimo Niskanen, Erlang/OTP, Ericsson AB >>>> >>>> ________________________________________________________________ >>>> erlang-bugs mailing list. See http://www.erlang.org/faq.html >>>> erlang-bugs (at) erlang.org >>>> >>>> >>> >>> > > >> #include >> #include >> #include >> #include >> #include >> #include >> #include >> >> void use_inet_addr(const char *name); >> void use_inet_aton(const char *name); >> void use_inet_pton(const char *name, int af); >> void use_gethostbyname(const char *name); >> void use_gethostbyname_r(const char *name); >> void use_gethostbyname2(const char *name, int af); >> void use_gethostbyname2_r(const char *name, int af); >> void use_getaddrinfo(const char *name); >> >> int main(int argc, char *argv[]) >> { >> if(argc != 2) { >> printf("Usage: %s \n", argv[0]); >> return 1; >> } >> >> use_inet_addr(argv[1]); >> use_inet_aton(argv[1]); >> use_inet_pton(argv[1], AF_INET); >> use_gethostbyname(argv[1]); >> use_gethostbyname_r(argv[1]); >> #if defined(_BSD_SOURCE) || defined(_SVID_SOURCE) >> use_gethostbyname2(argv[1], AF_INET); >> use_gethostbyname2_r(argv[1], AF_INET); >> #endif >> use_getaddrinfo(argv[1]); >> >> return 0; >> } >> >> void use_inet_addr(const char *name) >> { >> in_addr_t addr = inet_addr(name); >> >> printf("inet_addr: "); >> if(addr == INADDR_NONE) { >> printf("failed (possibly 255.255.255.255)\n"); >> } else { >> struct in_addr in; >> in.s_addr = addr; >> printf("%s\n", inet_ntoa(in)); >> } >> } >> >> void use_inet_aton(const char *name) >> { >> struct in_addr in; >> >> printf("inet_aton: "); >> if(inet_aton(name, &in)) { >> printf("%s\n", inet_ntoa(in)); >> } else { >> printf("failed\n"); >> } >> } >> >> void use_inet_pton(const char *name, int af) >> { >> struct in_addr in; >> >> printf("inet_pton: "); >> if(inet_pton(af, name, &in)) { >> printf("%s\n", inet_ntoa(in)); >> } else { >> printf("failed\n"); >> } >> } >> >> void use_gethostbyname(const char *name) >> { >> struct hostent *h = gethostbyname(name); >> >> printf("gethostbyname: "); >> if(!h) { >> printf("failed (%s)\n", hstrerror(h_errno)); >> } else { >> if(h->h_addrtype != AF_INET) { >> printf("failed (invalid address type)\n"); >> } else { >> char **pp = h->h_addr_list; >> while(*pp != NULL) { >> struct in_addr *p = (struct in_addr*)(*pp); >> printf("%s\n", inet_ntoa(*p)); >> ++pp; >> } >> } >> } >> } >> >> void use_gethostbyname_r(const char *name) >> { >> int rc; >> char buf[8192]; >> struct hostent h; >> struct hostent *rp; >> int myerrno; >> >> printf("gethostbyname_r: "); >> rc = gethostbyname_r(name, &h, buf, sizeof(buf), &rp, &myerrno); >> if(rc == ERANGE) { >> printf("failed (out of memory)\n"); >> } else if(!rc) { >> if(!rp) { >> printf("no address found\n"); >> } else { >> char **pp = h.h_addr_list; >> while(*pp != NULL) { >> struct in_addr *p = (struct in_addr*)(*pp); >> printf("%s\n", inet_ntoa(*p)); >> ++pp; >> } >> } >> } else { >> printf("failed (%s)\n", hstrerror(myerrno)); >> } >> } >> >> #if defined(_BSD_SOURCE) || defined(_SVID_SOURCE) >> >> void use_gethostbyname2(const char *name, int af) >> { >> struct hostent *h = gethostbyname2(name, af); >> >> printf("gethostbyname2: "); >> if(!h) { >> printf("failed (%s)\n", hstrerror(h_errno)); >> } else { >> if(h->h_addrtype != AF_INET) { >> printf("failed (invalid address type)\n"); >> } else { >> char **pp = h->h_addr_list; >> while(*pp != NULL) { >> struct in_addr *p = (struct in_addr*)(*pp); >> printf("%s\n", inet_ntoa(*p)); >> ++pp; >> } >> } >> } >> } >> >> void use_gethostbyname2_r(const char *name, int af) >> { >> int rc; >> char buf[8192]; >> struct hostent h; >> struct hostent *rp; >> int myerrno; >> >> printf("gethostbyname2_r: "); >> rc = gethostbyname2_r(name, af, &h, buf, sizeof(buf), &rp, &myerrno); >> if(rc == ERANGE) { >> printf("failed (out of memory)\n"); >> } else if(!rc) { >> if(!rp) { >> printf("no address found\n"); >> } else { >> char **pp = h.h_addr_list; >> while(*pp != NULL) { >> struct in_addr *p = (struct in_addr*)(*pp); >> printf("%s\n", inet_ntoa(*p)); >> ++pp; >> } >> } >> } else { >> printf("failed (%s)\n", hstrerror(myerrno)); >> } >> } >> >> #endif >> >> void use_getaddrinfo(const char *name) >> { >> int rc; >> struct addrinfo hints; >> struct addrinfo *resp; >> >> printf("getaddrinfo: "); >> >> memset(&hints, 0, sizeof(hints)); >> hints.ai_family = AF_INET; >> hints.ai_flags = AI_ADDRCONFIG | AI_PASSIVE; >> hints.ai_socktype = SOCK_STREAM; >> >> rc = getaddrinfo(name, NULL, &hints, &resp); >> if(rc) { >> printf("failed (%s)\n", gai_strerror(rc)); >> } else { >> struct addrinfo *rp; >> >> for(rp = resp; rp != NULL; rp = rp->ai_next) { >> struct sockaddr_in *addr = (struct sockaddr_in*)(rp->ai_addr); >> printf("%s\n", inet_ntoa(addr->sin_addr)); >> } >> >> freeaddrinfo(resp); >> } >> } >> >> >> > > >> ________________________________________________________________ >> erlang-bugs mailing list. See http://www.erlang.org/faq.html >> erlang-bugs (at) erlang.org >> > > From roberto.aloi@REDACTED Wed Jan 20 20:23:27 2010 From: roberto.aloi@REDACTED (Roberto Aloi) Date: Wed, 20 Jan 2010 19:23:27 +0000 (GMT) Subject: This mysterious ssh application Message-ID: <2126240860.58971264015407955.JavaMail.root@zimbra> Hi all, I was trying to play a bit with the ssh application and I found some fancy stuff in it. First of all, I've noticed that the public_key application (required by ssh) was not in the applications list of the ssh application (in the .app.src, I mean). Since the public_key application was not loaded in my system, it took me a while to understand this (the error in the stack trace was showing something cryptic like: "{no_io_allowed,read_password}"). I've already created and submitted a simple patch on github. I must admit I've read too late the guidelines for submitting patches to Erlang OTP, so I will try to be a better boy the next time. Second and most important. Reading from the documentation, it seems I can specify a callback module for my SSH connection: ---BEGIN CITE--- {key_cb, atom() = KeyCallbackModule} Provide a special call-back module for key handling. The call-back module should be modeled after the ssh_file module. The functions that must be exported are: private_host_rsa_key/2, private_host_dsa_key/2, lookup_host_key/3 and add_host_key/3. This is considered somewhat experimental and will be better documented later on. ---END CITE--- Well, the ssh_file should be a behaviour and it's not. The functions that are present in the documentation as the ones to be exported seems not to be enough (at least for my app). At least private_identity_key/2 should be included. Functions like the ssh_auth:publickey_msg/1 refer to the ssh_file module directly, while they should use functions defined in the callback module, if needed. I actually needed a parametrized module, so I couldn't specify an atom for my key_cb option. Still, I passed the checks and my parameter has been accepted. That was good, but the documentation should be corrected. I know all this is meant to be an experimental part of the SSH application, but I would like to know if anyone is working on that. I'll hopefully try to fix some of these as soon I find some spare hours. Best regards, -- Roberto Aloi Twitter: @prof3ta Blog: http://aloiroberto.wordpress.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From igorrs@REDACTED Sun Jan 24 08:51:38 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sun, 24 Jan 2010 05:51:38 -0200 Subject: Mnesia deadlocks (?) while loading table Message-ID: I have been able to reproduce this some times on a 64-bit Ubuntu, a 64-bit CentOS and a 32-bit Ubuntu, running R13B02. Open 2 terminals and start one node on each: erl -sname test1 erl -sname test2 Create the schema from the first node (substitute ijaba2 with your server's name): (test1@REDACTED)1> ok = mnesia:create_schema([node(), test2@REDACTED]), mnesia:start(). Start Mnesia also on the second node: (test2@REDACTED)1> mnesia:start(). Create a test table from the first node and start writing to it: (test1@REDACTED)2> mnesia:create_table(test, [{disc_only_copies, mnesia:system_info(running_db_nodes)}]). (test1@REDACTED)3> W = fun(F, N) -> mnesia:sync_transaction(fun mnesia:write/1, [{test, N, N}]), F(F, N + 1) end, W(W, 1). Restart Mnesia on the second node and wait for the table to be loaded: (test2@REDACTED)2> mnesia:stop(), ok = mnesia:start(). (test2@REDACTED)3> mnesia:wait_for_tables([test], infinity). For some runs of this experiment, the table will never load. It seems the current writer transaction on the first node is waiting for a commit, while holding a write lock: (test3@REDACTED)1> rpc:call(test1@REDACTED, mnesia, system_info, [held_locks]). [{{test,14691},write,{tid,14696,<6217.41.0>}}] (test3@REDACTED)3> rpc:call(test1@REDACTED, erlang, process_info, [list_to_pid("<6217.41.0>"), current_function]). {current_function,{mnesia_tm,rec_all,4}} The second node is waiting for the table to be received (test3@REDACTED)4> rpc:call(test2@REDACTED, mnesia, system_info, [held_locks]). [{{schema,test},read,{tid,14009,<6358.199.0>}}] (test3@REDACTED)5> rpc:call(test2@REDACTED, erlang, process_info, [list_to_pid("<6358.199.0>"), current_function]). {current_function,{mnesia_loader,wait_on_load_complete,1}} And there's a third transaction going on (the table sender?): (test3@REDACTED)7> rpc:call(test1@REDACTED, mnesia, system_info, [transactions]). [{14696,<6217.41.0>,coordinator}, {14697,<6217.176.0>,coordinator}] (test3@REDACTED)8> rpc:call(test1@REDACTED, erlang, process_info, [list_to_pid("<6217.176.0>"), current_function]). {current_function,{timer,sleep,1}} None of the 3 transactions makes any progress, so maybe there's a circular waiting here (deadlock). I hope you can reproduce it easily. It seems to depend on whether the first node notices test2's restart before or after test2 starts loading the table. But maybe I'm wrong and there's another race condition. Igor. -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From igorrs@REDACTED Sun Jan 24 17:48:41 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Sun, 24 Jan 2010 14:48:41 -0200 Subject: Mnesia deadlocks (?) while loading table In-Reply-To: References: Message-ID: It's not so easy as I thought to reproduce the problem with only one table. I am attaching code that creates more tables and executes transactions with two of them. This is not a necessary condition to reproduce the problem, but it helps a lot. Open 2 terminals and start one node on each: erl -sname test1 erl -sname test2 Create everything from the first node (substitute igorrs with your server's name): (test1@REDACTED)1> load_dl:start(test2@REDACTED). Run this restart loop on second node: (test2@REDACTED)1> load_dl:restart_forever(). At some point, the second node will stop printing and the last message will be "Started. Waiting for tables." It will be hung forever, not being able to load the tables. At the same time, the current transaction on the first node will also be hung forever (I got some info - see below - by running some RPCs from a third node). Igor. On Sun, Jan 24, 2010 at 5:51 AM, Igor Ribeiro Sucupira wrote: > I have been able to reproduce this some times on a 64-bit Ubuntu, a > 64-bit CentOS and a 32-bit Ubuntu, running R13B02. > > Open 2 terminals and start one node on each: > erl -sname test1 > erl -sname test2 > Create the schema from the first node (substitute ijaba2 with your > server's name): > (test1@REDACTED)1> ok = mnesia:create_schema([node(), test2@REDACTED]), > mnesia:start(). > Start Mnesia also on the second node: > (test2@REDACTED)1> mnesia:start(). > Create a test table from the first node and start writing to it: > (test1@REDACTED)2> mnesia:create_table(test, [{disc_only_copies, > mnesia:system_info(running_db_nodes)}]). > (test1@REDACTED)3> W = fun(F, N) -> mnesia:sync_transaction(fun > mnesia:write/1, [{test, N, N}]), F(F, N + 1) end, W(W, 1). > Restart Mnesia on the second node and wait for the table to be loaded: > (test2@REDACTED)2> mnesia:stop(), ok = mnesia:start(). > (test2@REDACTED)3> mnesia:wait_for_tables([test], infinity). > > > For some runs of this experiment, the table will never load. > It seems the current writer transaction on the first node is waiting > for a commit, while holding a write lock: > (test3@REDACTED)1> rpc:call(test1@REDACTED, mnesia, system_info, [held_locks]). > [{{test,14691},write,{tid,14696,<6217.41.0>}}] > (test3@REDACTED)3> rpc:call(test1@REDACTED, erlang, process_info, > [list_to_pid("<6217.41.0>"), current_function]). > {current_function,{mnesia_tm,rec_all,4}} > > The second node is waiting for the table to be received > (test3@REDACTED)4> rpc:call(test2@REDACTED, mnesia, system_info, > [held_locks]). > [{{schema,test},read,{tid,14009,<6358.199.0>}}] > (test3@REDACTED)5> rpc:call(test2@REDACTED, erlang, process_info, > [list_to_pid("<6358.199.0>"), current_function]). > {current_function,{mnesia_loader,wait_on_load_complete,1}} > > And there's a third transaction going on (the table sender?): > (test3@REDACTED)7> rpc:call(test1@REDACTED, mnesia, system_info, [transactions]). > [{14696,<6217.41.0>,coordinator}, > ?{14697,<6217.176.0>,coordinator}] > (test3@REDACTED)8> rpc:call(test1@REDACTED, erlang, process_info, > [list_to_pid("<6217.176.0>"), current_function]). > {current_function,{timer,sleep,1}} > > None of the 3 transactions makes any progress, so maybe there's a > circular waiting here (deadlock). > > I hope you can reproduce it easily. It seems to depend on whether the > first node notices test2's restart before or after test2 starts > loading the table. But maybe I'm wrong and there's another race > condition. > > Igor. > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. -------------- next part -------------- A non-text attachment was scrubbed... Name: load_dl.erl Type: text/x-erlang Size: 1226 bytes Desc: not available URL: From dangud@REDACTED Mon Jan 25 08:39:55 2010 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 25 Jan 2010 08:39:55 +0100 Subject: [erlang-bugs] Re: Mnesia deadlocks (?) while loading table In-Reply-To: References: Message-ID: <93df43b61001242339n48b09336pbc8954eaa8440edf@mail.gmail.com> Thanks, I'll have a look at it. I'm on a hunt for a deadlock I can't reproduce... /Dan On Sun, Jan 24, 2010 at 5:48 PM, Igor Ribeiro Sucupira wrote: > It's not so easy as I thought to reproduce the problem with only one table. > I am attaching code that creates more tables and executes transactions > with two of them. This is not a necessary condition to reproduce the > problem, but it helps a lot. > > Open 2 terminals and start one node on each: > erl -sname test1 > erl -sname test2 > Create everything from the first node (substitute igorrs with your > server's name): > (test1@REDACTED)1> load_dl:start(test2@REDACTED). > Run this restart loop on second node: > (test2@REDACTED)1> load_dl:restart_forever(). > > At some point, the second node will stop printing and the last message > will be "Started. Waiting for tables." It will be hung forever, not > being able to load the tables. At the same time, the current > transaction on the first node will also be hung forever (I got some > info - see below - by running some RPCs from a third node). > > Igor. > > On Sun, Jan 24, 2010 at 5:51 AM, Igor Ribeiro Sucupira wrote: >> I have been able to reproduce this some times on a 64-bit Ubuntu, a >> 64-bit CentOS and a 32-bit Ubuntu, running R13B02. >> >> Open 2 terminals and start one node on each: >> erl -sname test1 >> erl -sname test2 >> Create the schema from the first node (substitute ijaba2 with your >> server's name): >> (test1@REDACTED)1> ok = mnesia:create_schema([node(), test2@REDACTED]), >> mnesia:start(). >> Start Mnesia also on the second node: >> (test2@REDACTED)1> mnesia:start(). >> Create a test table from the first node and start writing to it: >> (test1@REDACTED)2> mnesia:create_table(test, [{disc_only_copies, >> mnesia:system_info(running_db_nodes)}]). >> (test1@REDACTED)3> W = fun(F, N) -> mnesia:sync_transaction(fun >> mnesia:write/1, [{test, N, N}]), F(F, N + 1) end, W(W, 1). >> Restart Mnesia on the second node and wait for the table to be loaded: >> (test2@REDACTED)2> mnesia:stop(), ok = mnesia:start(). >> (test2@REDACTED)3> mnesia:wait_for_tables([test], infinity). >> >> >> For some runs of this experiment, the table will never load. >> It seems the current writer transaction on the first node is waiting >> for a commit, while holding a write lock: >> (test3@REDACTED)1> rpc:call(test1@REDACTED, mnesia, system_info, [held_locks]). >> [{{test,14691},write,{tid,14696,<6217.41.0>}}] >> (test3@REDACTED)3> rpc:call(test1@REDACTED, erlang, process_info, >> [list_to_pid("<6217.41.0>"), current_function]). >> {current_function,{mnesia_tm,rec_all,4}} >> >> The second node is waiting for the table to be received >> (test3@REDACTED)4> rpc:call(test2@REDACTED, mnesia, system_info, >> [held_locks]). >> [{{schema,test},read,{tid,14009,<6358.199.0>}}] >> (test3@REDACTED)5> rpc:call(test2@REDACTED, erlang, process_info, >> [list_to_pid("<6358.199.0>"), current_function]). >> {current_function,{mnesia_loader,wait_on_load_complete,1}} >> >> And there's a third transaction going on (the table sender?): >> (test3@REDACTED)7> rpc:call(test1@REDACTED, mnesia, system_info, [transactions]). >> [{14696,<6217.41.0>,coordinator}, >> ?{14697,<6217.176.0>,coordinator}] >> (test3@REDACTED)8> rpc:call(test1@REDACTED, erlang, process_info, >> [list_to_pid("<6217.176.0>"), current_function]). >> {current_function,{timer,sleep,1}} >> >> None of the 3 transactions makes any progress, so maybe there's a >> circular waiting here (deadlock). >> >> I hope you can reproduce it easily. It seems to depend on whether the >> first node notices test2's restart before or after test2 starts >> loading the table. But maybe I'm wrong and there's another race >> condition. >> >> Igor. >> >> -- >> "The secret of joy in work is contained in one word - excellence. To >> know how to do something well is to enjoy it." - Pearl S. Buck. >> > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org > From dgud@REDACTED Mon Jan 25 14:00:55 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Mon, 25 Jan 2010 14:00:55 +0100 Subject: [erlang-bugs] Re: Mnesia deadlocks (?) while loading table In-Reply-To: <93df43b61001242339n48b09336pbc8954eaa8440edf@mail.gmail.com> References: <93df43b61001242339n48b09336pbc8954eaa8440edf@mail.gmail.com> Message-ID: <93df43b61001250500m23fa9df3ndbb606da721b4fb6@mail.gmail.com> Here is a patch: diff --git a/lib/mnesia/src/mnesia_tm.erl b/lib/mnesia/src/mnesia_tm.erl index 3f3a10a..5a2407d 100644 --- a/lib/mnesia/src/mnesia_tm.erl +++ b/lib/mnesia/src/mnesia_tm.erl @@ -1388,7 +1388,9 @@ multi_commit(sync_sym_trans, Tid, CR, Store) -> {WaitFor, Local} = ask_commit(sync_sym_trans, Tid, CR, DiscNs, RamNs), {Outcome, []} = rec_all(WaitFor, Tid, do_commit, []), ?eval_debug_fun({?MODULE, multi_commit_sym_sync}, - [{tid, Tid}, {outcome, Outcome}]), + [{tid, Tid}, {outcome, Outcome}]), + [?ets_insert(Store, {waiting_for_commit_ack, Node}) || + Node <- WaitFor], rpc:abcast(DiscNs -- [node()], ?MODULE, {Tid, Outcome}), rpc:abcast(RamNs -- [node()], ?MODULE, {Tid, Outcome}), case Outcome of /Dan On Mon, Jan 25, 2010 at 8:39 AM, Dan Gudmundsson wrote: > Thanks, I'll have a look at it. > I'm on a hunt for a deadlock I can't reproduce... > > /Dan > > On Sun, Jan 24, 2010 at 5:48 PM, Igor Ribeiro Sucupira wrote: >> It's not so easy as I thought to reproduce the problem with only one table. >> I am attaching code that creates more tables and executes transactions >> with two of them. This is not a necessary condition to reproduce the >> problem, but it helps a lot. >> >> Open 2 terminals and start one node on each: >> erl -sname test1 >> erl -sname test2 >> Create everything from the first node (substitute igorrs with your >> server's name): >> (test1@REDACTED)1> load_dl:start(test2@REDACTED). >> Run this restart loop on second node: >> (test2@REDACTED)1> load_dl:restart_forever(). >> >> At some point, the second node will stop printing and the last message >> will be "Started. Waiting for tables." It will be hung forever, not >> being able to load the tables. At the same time, the current >> transaction on the first node will also be hung forever (I got some >> info - see below - by running some RPCs from a third node). >> >> Igor. >> >> On Sun, Jan 24, 2010 at 5:51 AM, Igor Ribeiro Sucupira wrote: >>> I have been able to reproduce this some times on a 64-bit Ubuntu, a >>> 64-bit CentOS and a 32-bit Ubuntu, running R13B02. >>> >>> Open 2 terminals and start one node on each: >>> erl -sname test1 >>> erl -sname test2 >>> Create the schema from the first node (substitute ijaba2 with your >>> server's name): >>> (test1@REDACTED)1> ok = mnesia:create_schema([node(), test2@REDACTED]), >>> mnesia:start(). >>> Start Mnesia also on the second node: >>> (test2@REDACTED)1> mnesia:start(). >>> Create a test table from the first node and start writing to it: >>> (test1@REDACTED)2> mnesia:create_table(test, [{disc_only_copies, >>> mnesia:system_info(running_db_nodes)}]). >>> (test1@REDACTED)3> W = fun(F, N) -> mnesia:sync_transaction(fun >>> mnesia:write/1, [{test, N, N}]), F(F, N + 1) end, W(W, 1). >>> Restart Mnesia on the second node and wait for the table to be loaded: >>> (test2@REDACTED)2> mnesia:stop(), ok = mnesia:start(). >>> (test2@REDACTED)3> mnesia:wait_for_tables([test], infinity). >>> >>> >>> For some runs of this experiment, the table will never load. >>> It seems the current writer transaction on the first node is waiting >>> for a commit, while holding a write lock: >>> (test3@REDACTED)1> rpc:call(test1@REDACTED, mnesia, system_info, [held_locks]). >>> [{{test,14691},write,{tid,14696,<6217.41.0>}}] >>> (test3@REDACTED)3> rpc:call(test1@REDACTED, erlang, process_info, >>> [list_to_pid("<6217.41.0>"), current_function]). >>> {current_function,{mnesia_tm,rec_all,4}} >>> >>> The second node is waiting for the table to be received >>> (test3@REDACTED)4> rpc:call(test2@REDACTED, mnesia, system_info, >>> [held_locks]). >>> [{{schema,test},read,{tid,14009,<6358.199.0>}}] >>> (test3@REDACTED)5> rpc:call(test2@REDACTED, erlang, process_info, >>> [list_to_pid("<6358.199.0>"), current_function]). >>> {current_function,{mnesia_loader,wait_on_load_complete,1}} >>> >>> And there's a third transaction going on (the table sender?): >>> (test3@REDACTED)7> rpc:call(test1@REDACTED, mnesia, system_info, [transactions]). >>> [{14696,<6217.41.0>,coordinator}, >>> ?{14697,<6217.176.0>,coordinator}] >>> (test3@REDACTED)8> rpc:call(test1@REDACTED, erlang, process_info, >>> [list_to_pid("<6217.176.0>"), current_function]). >>> {current_function,{timer,sleep,1}} >>> >>> None of the 3 transactions makes any progress, so maybe there's a >>> circular waiting here (deadlock). >>> >>> I hope you can reproduce it easily. It seems to depend on whether the >>> first node notices test2's restart before or after test2 starts >>> loading the table. But maybe I'm wrong and there's another race >>> condition. >>> >>> Igor. >>> >>> -- >>> "The secret of joy in work is contained in one word - excellence. To >>> know how to do something well is to enjoy it." - Pearl S. Buck. >>> >> >> >> >> -- >> "The secret of joy in work is contained in one word - excellence. To >> know how to do something well is to enjoy it." - Pearl S. Buck. >> >> >> ________________________________________________________________ >> erlang-bugs mailing list. See http://www.erlang.org/faq.html >> erlang-bugs (at) erlang.org >> > From raimo+erlang-bugs@REDACTED Mon Jan 25 17:10:16 2010 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Mon, 25 Jan 2010 17:10:16 +0100 Subject: [erlang-bugs] possibly incorrect search order in inet:gethostbyname_tm/4 In-Reply-To: <4B571E91.204@gmail.com> References: <4B53CE31.6000904@gmail.com> <20100118150724.GB10908@erix.ericsson.se> <20100119122628.GA20697@erix.ericsson.se> <4B55DC30.8080302@gmail.com> <20100120112322.GA8363@erix.ericsson.se> <4B571E91.204@gmail.com> Message-ID: <20100125161016.GA26836@erix.ericsson.se> I have tossed the suggestion around, and it bounced a little... The reason we introduced the native resolver was to be bug-compatible with the underlying OS. This means we can not let IP string detection override the native resolver. There is even a fallback short circuit in the code today that prevents inet:gethostbyname from trying the IP string alternative after the native resolver has failed. This leaves us some other alternatives: 1) To extend the 'file' or the 'dns' lookup methods with IP string detection. 2) To do only do IP string detection when the 'native' lookup method is not used, but to do it first instead of last as today, within the inet:gethostbyname lookup method wrapper itself. 3) To introduce a new lookup method e.g 'ipstring' that the user can insert in the lookup chain wherever suitable. I do not like 1) since it would make either of those lookup methods impure, harder to test and harder to explain. I prefer 2) because it would behave kind of natural. However, 3) is the purest but would require manual configuration in more cases. But 2) wins... At least until someone convinces me otherwise. On Wed, Jan 20, 2010 at 11:17:37PM +0800, Chaos Wang wrote: > Raimo Niskanen wrote: > >On Wed, Jan 20, 2010 at 12:22:08AM +0800, Chaos Wang wrote: > > > >>Cool~ > >> > >>Sorry for responding so late. I was digging into some glibc source code... > >> > >>The followings are my findings (IPv4 only on Linux). And I totally agree > >>with you, that the safest form to be considered as a IPv4 address will > >>be the standard IPv4 dotted-decimal notation without trailing dot. > >> > > > >But I think the best way would be to adopt the Solaris/Linux behaviour. > >Perhaps even also the hex/octal notation... > > > Either way is OK to me, as long as it can recognize addresses in > standard IPv4 dotted-decimal notation correctly ;-) > > > >>All libc APIs related to parsing IPv4 address string into in_addr form > >>in my mind are: > >> * inet_addr() (deprecated) > >> * inet_aton() > >> * inet_pton() > >> * gethostbyname() and gethostbyname_r() (obsolete, but used by > >>inet_gethost program) > >> * gethostbyname2() and gethostbyname2_r() (GNU extension) > >> * getaddrinfo() > >> > >>In all these functions, strings with trailing dot will not be considered > >>as IPv4 addresses. > >> > >>inet_aton() (and deprecated inet_addr()) recognize IPv4 numbers-and-dots > >>notation: every dotted number in the address can be in decimal, octal or > >>hexadecimal. And the address can also be written in shorthand: > >> > >> a - means treat a as 32 bits > >> a.b - means treat b as 24 bits > >> a.b.c - means treat c as 16 bits > >> > >>inet_pton() is like inet_aton(), but without all the hexadecimal, octal > >>(with the exception of 0) and shorthand. So it only recognizes standard > >>IPv4 dotted-decimal notation. > >> > >>gethostbyname() (also gethostbyname2() and *_r alternations) use > >>__nss_hostname_digits_dots() to identify IP address. This function calls > >>inet_aton() to parse IPv4 address, except that it refuse to accept any > >>non-digit characters. So the hexadecimal from of IPv4 addresses can't be > >>recognized by it. > >> > >>getaddrinfo() use inet_aton() to recognize IPv4 address. So they are > >>equivalent in IPv4 address parsing. > >> > > > >I just found out you have almost created the Solaris man page for > >inet_pton: > >http://www.s-gms.ms.edus.si/cgi-bin/man-cgi -> search command inet_pton > > > Ah, truly a coincidence :-) > Manpages in Solaris are more thorough than in Linux, indeed. > > > >>The program I used to test these APIs is in the attachments. > >> > >>Reference locations (in glibc-2.9): > >> * resolv/inet_addr.c implements inet_aton(), inet_addr() > >> * resolv/inet_pton.c implements inet_pton() > >> * sysdeps/posix/getaddrinfo.c implements getaddrinfo() > >> * nss/getXXbyYY.c implements gethostbyname*() > >> * nss/getXXbyYY_r.c implements gethostbyname*_r() > >> * nss/digits_dots.c implements __nss_hostname_digits_dots() > >> > >>Raimo Niskanen wrote: > >> > >>>I have done some research on my own... > >>> > >>>These are the ones that succeed (and other numbers > >>>within the ranges, of course): > >>> > >>>Linux, FreeBSD, Solaris: > >>> AF_INET > >>>"127.0.0.1" -> 127.0.0.1 > >>>"192.168.1 -> 192.168.0.1 > >>>"10.1" -> 10.0.0.1 > >>>"17" -> 0.0.0.17 > >>>"192.168.65535" -> 192.168.255.255 > >>>"10.16777215" -> 10.255.255.255 > >>>"4294967295" -> 255.255.255.255 > >>> AF_INET6 > >>>"127.0.0.1" -> ::ffff:127.0.0.1 > >>>"192.168.1 -> ::ffff:192.168.0.1 > >>>"10.1" -> ::ffff:10.0.0.1 > >>>"17" -> ::ffff:0.0.0.17 > >>>"192.168.65535" -> ::ffff:192.168.255.255 > >>>"10.16777215" -> ::ffff:10.255.255.255 > >>>"4294967295" -> ::ffff:255.255.255.255 > >>>"::127.0.0.1" -> ::127.0.0.1 > >>>"::" -> :: > >>> > >>>FreeBSD (addendum): > >>> AF_INET > >>>"127.0.0.1." -> 127.0.0.1 > >>> > >>>OpenBSD: > >>> AF_INET > >>>"127.0.0.1" -> 127.0.0.1 > >>>"127.0.0.1." -> 127.0.0.1 > >>> AF_INET6 > >>>"::127.0.0.1" -> ::127.0.0.1 > >>>"::" -> :: > >>> > >>>For IPv6 addresses there seems to be consensus: if it > >>>parses as an IPv6 address according to the specifications > >>>I recall, that IPv6 address is returned, except that OpenBSD > >>>does not accept an IPv4 string when requesting an IPv6 > >>>address while the others do. > >>> > >>>For IPv4 addresses Linux, FreeBSD and Solaris regards > >>>many numeric strings as IPv4 addresses while OpenBSD > >>>requires a 4-field dotted decimal. Both BSDs accept > >>>a trailing dot for 4-field dotted decimal, while > >>>Linux and Solaris regard a trailing dot as proof > >>>that the string is an absolute hostname. > >>> > >>>Conclusions: > >>> > >>>The least common denominator (and the most common case) > >>>would be to regard 4-field dotted decimal [0..255] > >>>with no trailing dot as an IPv4 string. > >>> > >>>The most widespread behaviour would be the Linux, Solaris > >>>and FreeBSD (except for trailing dot) behaviour. And since > >>>OpenBSD is not the origin of Erlang/OTP and has little > >>>importance in the community, it would probably be > >>>the most sensible behaviour. > >>> > >>>The current inet_parse:ipv4_address/1 needs to be > >>>augumented to handle the "192.168.65535", > >>>"10.16777215" and "4294967295" IPv4 strings. > >>> > >>>I'll toss the suggestion around internally and see if and when > >>>we can make such a change into the Linux/Solaris behaviour... > >>> > >>>/ Raimo > >>> > >>> > >>> > >>>On Mon, Jan 18, 2010 at 04:07:24PM +0100, Raimo Niskanen wrote: > >>> > >>> > >>>>On Mon, Jan 18, 2010 at 10:57:53AM +0800, Chaos Wang wrote: > >>>> > >>>> > >>>>>Hi all, > >>>>> > >>>>>inet:gethostbyname_tm/4 always try any specified DNS resolution > >>>>>methods first, and check whether the given domain name is a IPv4/v6 > >>>>>address when all previous tries failed. So even a string containing > >>>>>valid IP address is specified as domain name to be resolved, it still > >>>>>needs to traverse all the resolution methods before finding out it's > >>>>>already an IP address at last. > >>>>> > >>>>>This would cause serious problems if 'dns' resolution method is > >>>>>specified in some corporation internal networks, in which all unknown > >>>>>domain names (including those treated-as-domain IPv4/v6 address > >>>>>string) will be resolved into the same portal server address. Only > >>>>>'native' resolution method can be used in such an environment, because > >>>>>libc DNS resolving API will check whether the domain name is an IP > >>>>>address at first. > >>>>> > >>>>>For example, in my working network, the resolving results when > >>>>>specified {lookup,[native]} in kernel inetrc are as following: > >>>>> > >>>>> > inet:getaddr("www.google.com", inet). % real domain name, > >>>>>resolvable at DNS server > >>>>> {ok,{64,233,189,99}} > >>>>> > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP > >>>>>address, not resolvable at DNS server > >>>>> {ok,{10,0,0,2}} > >>>>> > >>>>>But when specified {lookup,[dns]} in kernel inetrc, the results became: > >>>>> > >>>>> > inet:getaddr("www.google.com", inet). % real domain name, > >>>>>resolvable at DNS server > >>>>> {ok,{64,233,189,99}} > >>>>> > inet:getaddr("10.0.0.2", inet). % treated-as-domain IP > >>>>>address, resolved to portal server address by DNS server > >>>>> {ok,{115,124,17,136}} % Oops... > >>>>> > >>>>>IMHO the search order in inet:gethostbyname_tm/4 should be changed to: > >>>>>checking whether the domain name is already a IP address firstly, then > >>>>>try all specified domain resolution methods. > >>>>> > >>>>>Thanks! > >>>>> > >>>>> > >>>>Hi! > >>>> > >>>>You make a good case for changing the resolving order. I am almost > >>>>on your side, there is just one little detail...: > >>>> > >>>>Historically, portal server fake IP addresses has not been an issue > >>>>for inet_res (the DNS resolver). Instead, it has had to balance between > >>>>the RFCs and what actually is done in product networks. > >>>> > >>>>It is not impossible for inet_res to be in an environment where > >>>>the default domain is foo.bar and a lookup for "17" is supposed > >>>>to return the IP address for the host 17.foo.bar. Now "17" is > >>>>not a DNS label according to RFC 1035 section 2.3.1 but that > >>>>is only a "Preferred name syntax". > >>>> > >>>>Today it is more unlikely. But the question still is; > >>>>when can you safely assume the lookup string at hand is > >>>>an IP address and not a host name. > >>>> > >>>>The existing function inet_parse:ipv4_address is probably > >>>>too forgiving since it translates "17" -> {0,0,0,17}, > >>>>"17.18" -> {17,0,0,18}, "17.18.19" -> {17,18,0,19} > >>>>and "17.18.19.20" -> {17,18,19,20}, all from ancient > >>>>praxis or even standards. > >>>> > >>>>IPv6 addresses are more clear cut since any IPv6 address must contain > >>>>at least two colons and that is very unlikely for a host name. > >>>> > >>>>Can you strengthen your case by finding out more what it takes for > >>>>libc DNS to be convinced the lookup string is an IPv4 address? > >>>> > >>>> > >>>> > >>>>>chaoslawful > >>>>> > >>>>> > >>>>>________________________________________________________________ > >>>>>erlang-bugs mailing list. See http://www.erlang.org/faq.html > >>>>>erlang-bugs (at) erlang.org > >>>>> > >>>>> > >>>>-- > >>>> > >>>>/ Raimo Niskanen, Erlang/OTP, Ericsson AB > >>>> > >>>>________________________________________________________________ > >>>>erlang-bugs mailing list. See http://www.erlang.org/faq.html > >>>>erlang-bugs (at) erlang.org > >>>> > >>>> > >>> > >>> > > > > > >>#include > >>#include > >>#include > >>#include > >>#include > >>#include > >>#include > >> > >>void use_inet_addr(const char *name); > >>void use_inet_aton(const char *name); > >>void use_inet_pton(const char *name, int af); > >>void use_gethostbyname(const char *name); > >>void use_gethostbyname_r(const char *name); > >>void use_gethostbyname2(const char *name, int af); > >>void use_gethostbyname2_r(const char *name, int af); > >>void use_getaddrinfo(const char *name); > >> > >>int main(int argc, char *argv[]) > >>{ > >> if(argc != 2) { > >> printf("Usage: %s \n", argv[0]); > >> return 1; > >> } > >> > >> use_inet_addr(argv[1]); > >> use_inet_aton(argv[1]); > >> use_inet_pton(argv[1], AF_INET); > >> use_gethostbyname(argv[1]); > >> use_gethostbyname_r(argv[1]); > >>#if defined(_BSD_SOURCE) || defined(_SVID_SOURCE) > >> use_gethostbyname2(argv[1], AF_INET); > >> use_gethostbyname2_r(argv[1], AF_INET); > >>#endif > >> use_getaddrinfo(argv[1]); > >> > >> return 0; > >>} > >> > >>void use_inet_addr(const char *name) > >>{ > >> in_addr_t addr = inet_addr(name); > >> > >> printf("inet_addr: "); > >> if(addr == INADDR_NONE) { > >> printf("failed (possibly 255.255.255.255)\n"); > >> } else { > >> struct in_addr in; > >> in.s_addr = addr; > >> printf("%s\n", inet_ntoa(in)); > >> } > >>} > >> > >>void use_inet_aton(const char *name) > >>{ > >> struct in_addr in; > >> > >> printf("inet_aton: "); > >> if(inet_aton(name, &in)) { > >> printf("%s\n", inet_ntoa(in)); > >> } else { > >> printf("failed\n"); > >> } > >>} > >> > >>void use_inet_pton(const char *name, int af) > >>{ > >> struct in_addr in; > >> > >> printf("inet_pton: "); > >> if(inet_pton(af, name, &in)) { > >> printf("%s\n", inet_ntoa(in)); > >> } else { > >> printf("failed\n"); > >> } > >>} > >> > >>void use_gethostbyname(const char *name) > >>{ > >> struct hostent *h = gethostbyname(name); > >> > >> printf("gethostbyname: "); > >> if(!h) { > >> printf("failed (%s)\n", hstrerror(h_errno)); > >> } else { > >> if(h->h_addrtype != AF_INET) { > >> printf("failed (invalid address type)\n"); > >> } else { > >> char **pp = h->h_addr_list; > >> while(*pp != NULL) { > >> struct in_addr *p = (struct in_addr*)(*pp); > >> printf("%s\n", inet_ntoa(*p)); > >> ++pp; > >> } > >> } > >> } > >>} > >> > >>void use_gethostbyname_r(const char *name) > >>{ > >> int rc; > >> char buf[8192]; > >> struct hostent h; > >> struct hostent *rp; > >> int myerrno; > >> > >> printf("gethostbyname_r: "); > >> rc = gethostbyname_r(name, &h, buf, sizeof(buf), &rp, &myerrno); > >> if(rc == ERANGE) { > >> printf("failed (out of memory)\n"); > >> } else if(!rc) { > >> if(!rp) { > >> printf("no address found\n"); > >> } else { > >> char **pp = h.h_addr_list; > >> while(*pp != NULL) { > >> struct in_addr *p = (struct in_addr*)(*pp); > >> printf("%s\n", inet_ntoa(*p)); > >> ++pp; > >> } > >> } > >> } else { > >> printf("failed (%s)\n", hstrerror(myerrno)); > >> } > >>} > >> > >>#if defined(_BSD_SOURCE) || defined(_SVID_SOURCE) > >> > >>void use_gethostbyname2(const char *name, int af) > >>{ > >> struct hostent *h = gethostbyname2(name, af); > >> > >> printf("gethostbyname2: "); > >> if(!h) { > >> printf("failed (%s)\n", hstrerror(h_errno)); > >> } else { > >> if(h->h_addrtype != AF_INET) { > >> printf("failed (invalid address type)\n"); > >> } else { > >> char **pp = h->h_addr_list; > >> while(*pp != NULL) { > >> struct in_addr *p = (struct in_addr*)(*pp); > >> printf("%s\n", inet_ntoa(*p)); > >> ++pp; > >> } > >> } > >> } > >>} > >> > >>void use_gethostbyname2_r(const char *name, int af) > >>{ > >> int rc; > >> char buf[8192]; > >> struct hostent h; > >> struct hostent *rp; > >> int myerrno; > >> > >> printf("gethostbyname2_r: "); > >> rc = gethostbyname2_r(name, af, &h, buf, sizeof(buf), &rp, &myerrno); > >> if(rc == ERANGE) { > >> printf("failed (out of memory)\n"); > >> } else if(!rc) { > >> if(!rp) { > >> printf("no address found\n"); > >> } else { > >> char **pp = h.h_addr_list; > >> while(*pp != NULL) { > >> struct in_addr *p = (struct in_addr*)(*pp); > >> printf("%s\n", inet_ntoa(*p)); > >> ++pp; > >> } > >> } > >> } else { > >> printf("failed (%s)\n", hstrerror(myerrno)); > >> } > >>} > >> > >>#endif > >> > >>void use_getaddrinfo(const char *name) > >>{ > >> int rc; > >> struct addrinfo hints; > >> struct addrinfo *resp; > >> > >> printf("getaddrinfo: "); > >> > >> memset(&hints, 0, sizeof(hints)); > >> hints.ai_family = AF_INET; > >> hints.ai_flags = AI_ADDRCONFIG | AI_PASSIVE; > >> hints.ai_socktype = SOCK_STREAM; > >> > >> rc = getaddrinfo(name, NULL, &hints, &resp); > >> if(rc) { > >> printf("failed (%s)\n", gai_strerror(rc)); > >> } else { > >> struct addrinfo *rp; > >> > >> for(rp = resp; rp != NULL; rp = rp->ai_next) { > >> struct sockaddr_in *addr = (struct > >> sockaddr_in*)(rp->ai_addr); > >> printf("%s\n", inet_ntoa(addr->sin_addr)); > >> } > >> > >> freeaddrinfo(resp); > >> } > >>} > >> > >> > >> > > > > > >>________________________________________________________________ > >>erlang-bugs mailing list. See http://www.erlang.org/faq.html > >>erlang-bugs (at) erlang.org > >> > > > > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From rvirding@REDACTED Mon Jan 25 18:00:21 2010 From: rvirding@REDACTED (Robert Virding) Date: Mon, 25 Jan 2010 18:00:21 +0100 Subject: [erlang-bugs] erl_scan counts white space badly In-Reply-To: <19284.22055.5653.320994@ornendil.du.uab.ericsson.se> References: <4B4CFFF8.5080107@erlang-consulting.com> <19284.22055.5653.320994@ornendil.du.uab.ericsson.se> Message-ID: <3dbc6d1c1001250900i79d0cdc8h840a3151d9b664d8@mail.gmail.com> 2010/1/18 Hans Bolinder : > [Ulf Wiger:] >> I tried the column tracking functionality of erl_scan: > ... >> The original source code wasn't misaligned, but the indentation >> used a mix of tabs and spaces. erl_scan treats all kinds of >> white space as one space. >> >> I think it would be reasonable to at the very least increment by >> 8 if it's a tab. Supporting a tab length option might also be >> possible, but then it would also be nice to have e.g. epp support >> passing of this option to the scanner. > > We will consider adding such an option in a future release. I would argue that it is not really the function of the scanner to do this type of thing as there are too many different options if it is to be supported properly. As the option 'return_white_spaces' exists it can very well be left to the caller of erl_scan. Robert From ulf.wiger@REDACTED Mon Jan 25 20:20:19 2010 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 25 Jan 2010 20:20:19 +0100 Subject: [erlang-bugs] erl_scan counts white space badly In-Reply-To: <3dbc6d1c1001250900i79d0cdc8h840a3151d9b664d8@mail.gmail.com> References: <4B4CFFF8.5080107@erlang-consulting.com> <19284.22055.5653.320994@ornendil.du.uab.ericsson.se> <3dbc6d1c1001250900i79d0cdc8h840a3151d9b664d8@mail.gmail.com> Message-ID: <4B5DEEF3.3020003@erlang-solutions.com> Robert Virding wrote: > 2010/1/18 Hans Bolinder : >> [Ulf Wiger:] >>> I tried the column tracking functionality of erl_scan: >> ... >>> The original source code wasn't misaligned, but the indentation >>> used a mix of tabs and spaces. erl_scan treats all kinds of >>> white space as one space. >>> >>> I think it would be reasonable to at the very least increment by >>> 8 if it's a tab. Supporting a tab length option might also be >>> possible, but then it would also be nice to have e.g. epp support >>> passing of this option to the scanner. >> We will consider adding such an option in a future release. > > I would argue that it is not really the function of the scanner to do > this type of thing as there are too many different options if it is to > be supported properly. As the option 'return_white_spaces' exists it > can very well be left to the caller of erl_scan. Well, as it currently works, it tracks the column number, so I wasn't asking for that particular feature to be added. But if it's going to track the column number, it should add whitespace correctly. It doesn't make sense to count tabs as one space, and I'd say it violates the Principle of Least Astonishment. In other words, a bug. :) Changing the current behaviour to counting tabs as 8 (hard-coded) would at least be an improvement. Making it configurable would be the extra feature. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From igorrs@REDACTED Tue Jan 26 00:55:05 2010 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Mon, 25 Jan 2010 21:55:05 -0200 Subject: [erlang-bugs] Re: Mnesia deadlocks (?) while loading table In-Reply-To: <93df43b61001250500m23fa9df3ndbb606da721b4fb6@mail.gmail.com> References: <93df43b61001242339n48b09336pbc8954eaa8440edf@mail.gmail.com> <93df43b61001250500m23fa9df3ndbb606da721b4fb6@mail.gmail.com> Message-ID: Thank you. The problem seems to be fixed with the patch (and I've also verified on the original system where I detected the problem). So, is this patch going to be in the next release? Best regards. Igor. On Mon, Jan 25, 2010 at 11:00 AM, Dan Gudmundsson wrote: > Here is a patch: > > diff --git a/lib/mnesia/src/mnesia_tm.erl b/lib/mnesia/src/mnesia_tm.erl > index 3f3a10a..5a2407d 100644 > --- a/lib/mnesia/src/mnesia_tm.erl > +++ b/lib/mnesia/src/mnesia_tm.erl > @@ -1388,7 +1388,9 @@ multi_commit(sync_sym_trans, Tid, CR, Store) -> > ? ? {WaitFor, Local} = ask_commit(sync_sym_trans, Tid, CR, DiscNs, RamNs), > ? ? {Outcome, []} = rec_all(WaitFor, Tid, do_commit, []), > ? ? ?eval_debug_fun({?MODULE, multi_commit_sym_sync}, > - ? ? ? ? ? ? ? ? ? [{tid, Tid}, {outcome, Outcome}]), > + ? ? ? ? ? ? ? ? ? [{tid, Tid}, {outcome, Outcome}]), > + ? ?[?ets_insert(Store, {waiting_for_commit_ack, Node}) || > + ? ? ? Node <- WaitFor], > ? ? rpc:abcast(DiscNs -- [node()], ?MODULE, {Tid, Outcome}), > ? ? rpc:abcast(RamNs -- [node()], ?MODULE, {Tid, Outcome}), > ? ? case Outcome of > > /Dan > > On Mon, Jan 25, 2010 at 8:39 AM, Dan Gudmundsson wrote: >> Thanks, I'll have a look at it. >> I'm on a hunt for a deadlock I can't reproduce... >> >> /Dan >> >> On Sun, Jan 24, 2010 at 5:48 PM, Igor Ribeiro Sucupira wrote: >>> It's not so easy as I thought to reproduce the problem with only one table. >>> I am attaching code that creates more tables and executes transactions >>> with two of them. This is not a necessary condition to reproduce the >>> problem, but it helps a lot. >>> >>> Open 2 terminals and start one node on each: >>> erl -sname test1 >>> erl -sname test2 >>> Create everything from the first node (substitute igorrs with your >>> server's name): >>> (test1@REDACTED)1> load_dl:start(test2@REDACTED). >>> Run this restart loop on second node: >>> (test2@REDACTED)1> load_dl:restart_forever(). >>> >>> At some point, the second node will stop printing and the last message >>> will be "Started. Waiting for tables." It will be hung forever, not >>> being able to load the tables. At the same time, the current >>> transaction on the first node will also be hung forever (I got some >>> info - see below - by running some RPCs from a third node). >>> >>> Igor. >>> >>> On Sun, Jan 24, 2010 at 5:51 AM, Igor Ribeiro Sucupira wrote: >>>> I have been able to reproduce this some times on a 64-bit Ubuntu, a >>>> 64-bit CentOS and a 32-bit Ubuntu, running R13B02. >>>> >>>> Open 2 terminals and start one node on each: >>>> erl -sname test1 >>>> erl -sname test2 >>>> Create the schema from the first node (substitute ijaba2 with your >>>> server's name): >>>> (test1@REDACTED)1> ok = mnesia:create_schema([node(), test2@REDACTED]), >>>> mnesia:start(). >>>> Start Mnesia also on the second node: >>>> (test2@REDACTED)1> mnesia:start(). >>>> Create a test table from the first node and start writing to it: >>>> (test1@REDACTED)2> mnesia:create_table(test, [{disc_only_copies, >>>> mnesia:system_info(running_db_nodes)}]). >>>> (test1@REDACTED)3> W = fun(F, N) -> mnesia:sync_transaction(fun >>>> mnesia:write/1, [{test, N, N}]), F(F, N + 1) end, W(W, 1). >>>> Restart Mnesia on the second node and wait for the table to be loaded: >>>> (test2@REDACTED)2> mnesia:stop(), ok = mnesia:start(). >>>> (test2@REDACTED)3> mnesia:wait_for_tables([test], infinity). >>>> >>>> >>>> For some runs of this experiment, the table will never load. >>>> It seems the current writer transaction on the first node is waiting >>>> for a commit, while holding a write lock: >>>> (test3@REDACTED)1> rpc:call(test1@REDACTED, mnesia, system_info, [held_locks]). >>>> [{{test,14691},write,{tid,14696,<6217.41.0>}}] >>>> (test3@REDACTED)3> rpc:call(test1@REDACTED, erlang, process_info, >>>> [list_to_pid("<6217.41.0>"), current_function]). >>>> {current_function,{mnesia_tm,rec_all,4}} >>>> >>>> The second node is waiting for the table to be received >>>> (test3@REDACTED)4> rpc:call(test2@REDACTED, mnesia, system_info, >>>> [held_locks]). >>>> [{{schema,test},read,{tid,14009,<6358.199.0>}}] >>>> (test3@REDACTED)5> rpc:call(test2@REDACTED, erlang, process_info, >>>> [list_to_pid("<6358.199.0>"), current_function]). >>>> {current_function,{mnesia_loader,wait_on_load_complete,1}} >>>> >>>> And there's a third transaction going on (the table sender?): >>>> (test3@REDACTED)7> rpc:call(test1@REDACTED, mnesia, system_info, [transactions]). >>>> [{14696,<6217.41.0>,coordinator}, >>>> ?{14697,<6217.176.0>,coordinator}] >>>> (test3@REDACTED)8> rpc:call(test1@REDACTED, erlang, process_info, >>>> [list_to_pid("<6217.176.0>"), current_function]). >>>> {current_function,{timer,sleep,1}} >>>> >>>> None of the 3 transactions makes any progress, so maybe there's a >>>> circular waiting here (deadlock). >>>> >>>> I hope you can reproduce it easily. It seems to depend on whether the >>>> first node notices test2's restart before or after test2 starts >>>> loading the table. But maybe I'm wrong and there's another race >>>> condition. >>>> >>>> Igor. >>>> >>>> -- >>>> "The secret of joy in work is contained in one word - excellence. To >>>> know how to do something well is to enjoy it." - Pearl S. Buck. >>>> >>> >>> >>> >>> -- >>> "The secret of joy in work is contained in one word - excellence. To >>> know how to do something well is to enjoy it." - Pearl S. Buck. >>> >>> >>> ________________________________________________________________ >>> erlang-bugs mailing list. See http://www.erlang.org/faq.html >>> erlang-bugs (at) erlang.org >>> >> > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From dgud@REDACTED Tue Jan 26 08:37:44 2010 From: dgud@REDACTED (Dan Gudmundsson) Date: Tue, 26 Jan 2010 08:37:44 +0100 Subject: [erlang-bugs] Re: Mnesia deadlocks (?) while loading table In-Reply-To: References: <93df43b61001242339n48b09336pbc8954eaa8440edf@mail.gmail.com> <93df43b61001250500m23fa9df3ndbb606da721b4fb6@mail.gmail.com> Message-ID: <93df43b61001252337r5c8bd9e5iba8e6d9b11e92e8@mail.gmail.com> Yes it will be applied. /Dan On Tue, Jan 26, 2010 at 12:55 AM, Igor Ribeiro Sucupira wrote: > Thank you. The problem seems to be fixed with the patch (and I've also > verified on the original system where I detected the problem). > > So, is this patch going to be in the next release? > > Best regards. > Igor. > > On Mon, Jan 25, 2010 at 11:00 AM, Dan Gudmundsson wrote: >> Here is a patch: >> >> diff --git a/lib/mnesia/src/mnesia_tm.erl b/lib/mnesia/src/mnesia_tm.erl >> index 3f3a10a..5a2407d 100644 >> --- a/lib/mnesia/src/mnesia_tm.erl >> +++ b/lib/mnesia/src/mnesia_tm.erl >> @@ -1388,7 +1388,9 @@ multi_commit(sync_sym_trans, Tid, CR, Store) -> >> ? ? {WaitFor, Local} = ask_commit(sync_sym_trans, Tid, CR, DiscNs, RamNs), >> ? ? {Outcome, []} = rec_all(WaitFor, Tid, do_commit, []), >> ? ? ?eval_debug_fun({?MODULE, multi_commit_sym_sync}, >> - ? ? ? ? ? ? ? ? ? [{tid, Tid}, {outcome, Outcome}]), >> + ? ? ? ? ? ? ? ? ? [{tid, Tid}, {outcome, Outcome}]), >> + ? ?[?ets_insert(Store, {waiting_for_commit_ack, Node}) || >> + ? ? ? Node <- WaitFor], >> ? ? rpc:abcast(DiscNs -- [node()], ?MODULE, {Tid, Outcome}), >> ? ? rpc:abcast(RamNs -- [node()], ?MODULE, {Tid, Outcome}), >> ? ? case Outcome of >> >> /Dan >> >> On Mon, Jan 25, 2010 at 8:39 AM, Dan Gudmundsson wrote: >>> Thanks, I'll have a look at it. >>> I'm on a hunt for a deadlock I can't reproduce... >>> >>> /Dan >>> >>> On Sun, Jan 24, 2010 at 5:48 PM, Igor Ribeiro Sucupira wrote: >>>> It's not so easy as I thought to reproduce the problem with only one table. >>>> I am attaching code that creates more tables and executes transactions >>>> with two of them. This is not a necessary condition to reproduce the >>>> problem, but it helps a lot. >>>> >>>> Open 2 terminals and start one node on each: >>>> erl -sname test1 >>>> erl -sname test2 >>>> Create everything from the first node (substitute igorrs with your >>>> server's name): >>>> (test1@REDACTED)1> load_dl:start(test2@REDACTED). >>>> Run this restart loop on second node: >>>> (test2@REDACTED)1> load_dl:restart_forever(). >>>> >>>> At some point, the second node will stop printing and the last message >>>> will be "Started. Waiting for tables." It will be hung forever, not >>>> being able to load the tables. At the same time, the current >>>> transaction on the first node will also be hung forever (I got some >>>> info - see below - by running some RPCs from a third node). >>>> >>>> Igor. >>>> >>>> On Sun, Jan 24, 2010 at 5:51 AM, Igor Ribeiro Sucupira wrote: >>>>> I have been able to reproduce this some times on a 64-bit Ubuntu, a >>>>> 64-bit CentOS and a 32-bit Ubuntu, running R13B02. >>>>> >>>>> Open 2 terminals and start one node on each: >>>>> erl -sname test1 >>>>> erl -sname test2 >>>>> Create the schema from the first node (substitute ijaba2 with your >>>>> server's name): >>>>> (test1@REDACTED)1> ok = mnesia:create_schema([node(), test2@REDACTED]), >>>>> mnesia:start(). >>>>> Start Mnesia also on the second node: >>>>> (test2@REDACTED)1> mnesia:start(). >>>>> Create a test table from the first node and start writing to it: >>>>> (test1@REDACTED)2> mnesia:create_table(test, [{disc_only_copies, >>>>> mnesia:system_info(running_db_nodes)}]). >>>>> (test1@REDACTED)3> W = fun(F, N) -> mnesia:sync_transaction(fun >>>>> mnesia:write/1, [{test, N, N}]), F(F, N + 1) end, W(W, 1). >>>>> Restart Mnesia on the second node and wait for the table to be loaded: >>>>> (test2@REDACTED)2> mnesia:stop(), ok = mnesia:start(). >>>>> (test2@REDACTED)3> mnesia:wait_for_tables([test], infinity). >>>>> >>>>> >>>>> For some runs of this experiment, the table will never load. >>>>> It seems the current writer transaction on the first node is waiting >>>>> for a commit, while holding a write lock: >>>>> (test3@REDACTED)1> rpc:call(test1@REDACTED, mnesia, system_info, [held_locks]). >>>>> [{{test,14691},write,{tid,14696,<6217.41.0>}}] >>>>> (test3@REDACTED)3> rpc:call(test1@REDACTED, erlang, process_info, >>>>> [list_to_pid("<6217.41.0>"), current_function]). >>>>> {current_function,{mnesia_tm,rec_all,4}} >>>>> >>>>> The second node is waiting for the table to be received >>>>> (test3@REDACTED)4> rpc:call(test2@REDACTED, mnesia, system_info, >>>>> [held_locks]). >>>>> [{{schema,test},read,{tid,14009,<6358.199.0>}}] >>>>> (test3@REDACTED)5> rpc:call(test2@REDACTED, erlang, process_info, >>>>> [list_to_pid("<6358.199.0>"), current_function]). >>>>> {current_function,{mnesia_loader,wait_on_load_complete,1}} >>>>> >>>>> And there's a third transaction going on (the table sender?): >>>>> (test3@REDACTED)7> rpc:call(test1@REDACTED, mnesia, system_info, [transactions]). >>>>> [{14696,<6217.41.0>,coordinator}, >>>>> ?{14697,<6217.176.0>,coordinator}] >>>>> (test3@REDACTED)8> rpc:call(test1@REDACTED, erlang, process_info, >>>>> [list_to_pid("<6217.176.0>"), current_function]). >>>>> {current_function,{timer,sleep,1}} >>>>> >>>>> None of the 3 transactions makes any progress, so maybe there's a >>>>> circular waiting here (deadlock). >>>>> >>>>> I hope you can reproduce it easily. It seems to depend on whether the >>>>> first node notices test2's restart before or after test2 starts >>>>> loading the table. But maybe I'm wrong and there's another race >>>>> condition. >>>>> >>>>> Igor. >>>>> >>>>> -- >>>>> "The secret of joy in work is contained in one word - excellence. To >>>>> know how to do something well is to enjoy it." - Pearl S. Buck. >>>>> >>>> >>>> >>>> >>>> -- >>>> "The secret of joy in work is contained in one word - excellence. To >>>> know how to do something well is to enjoy it." - Pearl S. Buck. >>>> >>>> >>>> ________________________________________________________________ >>>> erlang-bugs mailing list. See http://www.erlang.org/faq.html >>>> erlang-bugs (at) erlang.org >>>> >>> >> > > > > -- > "The secret of joy in work is contained in one word - excellence. To > know how to do something well is to enjoy it." - Pearl S. Buck. > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org > > From matthew@REDACTED Thu Jan 28 18:02:27 2010 From: matthew@REDACTED (Matthew Sackman) Date: Thu, 28 Jan 2010 17:02:27 +0000 Subject: win32sysinfo.c doesn't understand DWORDLONG is unsigned and long Message-ID: <20100128170226.GA4191@wellquite.org> void get_avail_mem_ext() { char answer[512]; MEMORYSTATUSEX ms; ms.dwLength=sizeof(MEMORYSTATUSEX); GlobalMemoryStatusEx(&ms); sprintf(answer,"%d %I64d %I64d %I64d %I64d %I64d %I64d\n", ms.dwMemoryLoad, ms.ullTotalPhys, ms.ullAvailPhys, ms.ullTotalPageFile, ms.ullAvailPageFile, ms.ullTotalVirtual, ms.ullAvailVirtual ); return_answer(answer); /* DWORD dwLength; DWORD dwMemoryLoad; DWORDLONG ullTotalPhys; DWORDLONG ullAvailPhys; DWORDLONG ullTotalPageFile; DWORDLONG ullAvailPageFile; DWORDLONG ullTotalVirtual; DWORDLONG ullAvailVirtual; */ } DWORD and DWORDLONG are unsigned and long. By printing with %I64d, you're reinterpreting them as signed ints, leading to windows users with 4GB of RAM being told they have negative amounts of RAM (or even 0). Eg: $ cat foo.c #include #include #include #include main() { uint64_t n = 4293308416; printf("%I64d\n", n); printf("%I64u\n", n); printf("%I64lu\n", n); return 0; } $ gcc -o foo foo.c && ./foo -1658880 4293308416 4293308416 Elsewhere in win32sysinfo.c, there are %I64u format specifiers. It looks like the only occurrence of %I64d is in the get_avail_mem_ext function. Because they're long, I *think* they should be %I64lu. I think what's happening is that the d makes it an int, thus the msb is set, hence the negative answer. And indeed, setting n to 4294967297 (i.e. 2^32 + 1) gives: 1 1 4294967297 Note that the above is running under a 64-bit Linux. I don't know if the behaviour of %I64 is the same under Windows. I also don't know what size an int is in a 32-bit application running on 64-bit windows. My C isn't all that great. My reading is that the %I64 just says to use the system locale, and to pad with whitespace up to 64 characters. Matthew From egil@REDACTED Thu Jan 28 18:59:14 2010 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 28 Jan 2010 18:59:14 +0100 Subject: [erlang-bugs] win32sysinfo.c doesn't understand DWORDLONG is unsigned and long In-Reply-To: <20100128170226.GA4191@wellquite.org> References: <20100128170226.GA4191@wellquite.org> Message-ID: <4B61D072.40304@erix.ericsson.se> Hi Matthew, Thank you for reporting this issue. I will have a look at this but I think you are correct. %I64d seems to be the wrong thing here. Regards, Bj?rn-Egil Erlang/OTP Matthew Sackman wrote: > void get_avail_mem_ext() { > char answer[512]; > MEMORYSTATUSEX ms; > ms.dwLength=sizeof(MEMORYSTATUSEX); > GlobalMemoryStatusEx(&ms); > sprintf(answer,"%d %I64d %I64d %I64d %I64d %I64d %I64d\n", > ms.dwMemoryLoad, > ms.ullTotalPhys, > ms.ullAvailPhys, > ms.ullTotalPageFile, > ms.ullAvailPageFile, > ms.ullTotalVirtual, > ms.ullAvailVirtual > ); > return_answer(answer); > /* > DWORD dwLength; > DWORD dwMemoryLoad; > DWORDLONG ullTotalPhys; > DWORDLONG ullAvailPhys; > DWORDLONG ullTotalPageFile; > DWORDLONG ullAvailPageFile; > DWORDLONG ullTotalVirtual; > DWORDLONG ullAvailVirtual; > */ > } > > DWORD and DWORDLONG are unsigned and long. By printing with %I64d, you're > reinterpreting them as signed ints, leading to windows users with 4GB of > RAM being told they have negative amounts of RAM (or even 0). Eg: > > $ cat foo.c > #include > #include > #include > #include > > main() > { > uint64_t n = 4293308416; > printf("%I64d\n", n); > printf("%I64u\n", n); > printf("%I64lu\n", n); > return 0; > } > > $ gcc -o foo foo.c && ./foo > -1658880 > 4293308416 > 4293308416 > > Elsewhere in win32sysinfo.c, there are %I64u format specifiers. It looks > like the only occurrence of %I64d is in the get_avail_mem_ext function. > > Because they're long, I *think* they should be %I64lu. I think what's > happening is that the d makes it an int, thus the msb is set, hence the > negative answer. And indeed, setting n to 4294967297 (i.e. 2^32 + 1) > gives: > > 1 > 1 > 4294967297 > > Note that the above is running under a 64-bit Linux. I don't know if the > behaviour of %I64 is the same under Windows. I also don't know what size > an int is in a 32-bit application running on 64-bit windows. My C isn't > all that great. My reading is that the %I64 just says to use the system > locale, and to pad with whitespace up to 64 characters. > > Matthew > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org >