From pguyot@REDACTED Tue Jun 1 17:30:54 2010 From: pguyot@REDACTED (Paul Guyot) Date: Tue, 1 Jun 2010 17:30:54 +0200 Subject: SSL 3.11.1 : Certificate-based authentication fails with Firefox Message-ID: <81392C6F-3920-4B03-884E-91342E216B18@kallisys.net> Hello, Certificate-based authentication fails with Firefox (on MacOS X), while the same setup succeeds with Safari, Chrome and curl (which probably use OpenSSL). The server is a very simple erlang SSL server that verifies the peer certificate : ssl:ssl_accept(ClientTransportSocket, [{ssl_imp, new}, {active, false}, {fail_if_no_peer_cert, true}, {verify, verify_peer}, {cacertfile, ?ALL_CERTIFICATES}, {certfile, ?SERVER_CERT_FILE}, {keyfile, ?SERVER_KEY_FILE}, {validate_extensions_fun, fun validate_extensions/4}, {verify_fun, fun(ErrorList) -> case ErrorList of [] -> true; _ -> io:format("ErrorList is ~p~n", [ErrorList]), false end end}], infinity) The bug seems to be in next_state/3, for the handshake case, around line 1766 of ssl_connection.erl (from the dev branch on github). {Packets, Buf} = ssl_handshake:get_tls_handshake(Data,Buf0, KeyAlg,Version), Start = {next_state, StateName, State0#state{tls_handshake_buffer = Buf}}, lists:foldl(Handle, Start, Packets) Handlers (invoked from Handle) can call next_record to get the next packet. For example, handle_peer_cert/3 (line 1148), is as follows : handle_peer_cert(PeerCert, PublicKeyInfo, #state{session = Session} = State0) -> State1 = State0#state{session = Session#session{peer_certificate = PeerCert}, public_key_info = PublicKeyInfo}, {Record, State} = next_record(State1), next_state(certify, Record, State). next_record/1 will then attempt to get the next record, ignoring Packets list: next_record(#state{tls_cipher_texts = [], socket = Socket} = State) -> inet:setopts(Socket, [{active,once}]), {no_record, State}; next_record(#state{tls_cipher_texts = [CT | Rest], connection_states = ConnStates0} = State) -> case ssl_record:decode_cipher_text(CT, ConnStates0) of {Plain, ConnStates} -> {Plain, State#state{tls_cipher_texts = Rest, connection_states = ConnStates}}; #alert{} = Alert -> {Alert, State} end. If the client sends a single packet in the handshake buffer (i.e. lists:foldl only runs once), everything is fine. The handler will call next_record that will either serve the packet from tls_cipher_texts or handshake with the transport layer with {active, once}. If the client sends more than one packet in the handshake buffer (Data + Buf0 above), and more data to the socket that can get returned by next_record, events end up being processed in wrong order. This is typically what happens with Firefox. Packets is composed of three elements : [{{certificate, [<<48,130,5,6,48,...>>]}, <<11, ...>>}, {{client_key_exchange, {client_diffie_hellman_public, <<...>>}}, <<16,...>>}, {{certificate_verify, <<120, ...>>}, <<15,...>>}] While at the same time, a record is present in tls_cipher_texts. Eventually, the process fails with a function_clause error because the state isn't what is expected (in this very case, there is no cipher_state defined here since no client_key_exchange packet was processed): ** Reason for termination = ** {function_clause,[{ssl_cipher,block_decipher, [#Fun,undefined,20, <<120,5,167,147,201,226,3,57,134,28,129,147, 41,178,143,254,181,2,49,7,140,13,191,3,135, 237,179,175,167,41,239,235,34,22,61,214, 228,77,124,198,115,53,25,153,151,63,51,197>>, {3,1}]}, {ssl_record,decipher,2}, {ssl_record,decode_cipher_text,2}, {ssl_connection,next_record,1}, {ssl_connection,next_state,3}, {lists,foldl,3}, <--- this is the lists:foldl above {ssl_connection,next_state,3}, {gen_fsm,handle_msg,7}]} Paul From a.zhuravlev@REDACTED Tue Jun 1 23:22:29 2010 From: a.zhuravlev@REDACTED (Alexander Zhuravlev) Date: Wed, 2 Jun 2010 01:22:29 +0400 Subject: Too strict HTTP Status Line parsing Message-ID: <20100601212229.GA58321@zaa.local> Hello, I've tried to use lhttpc library (http://bitbucket.org/etc/lhttpc) to fetch a resource (http://www.qype.com/review/1376848) and got the following error: {{http_error,"HTTP/1.1 200\r\n"}, [{lhttpc_client,read_response,5}, {lhttpc_client,execute,8}, {lhttpc_client,request,9}]} I've checked lhttpc source code and found out that to receive and parse an HTTP response it uses _standard_ erlang module gen_tcp on a socket in {packet, http} mode. So it looks like the {http_error,"HTTP/1.1 200\r\n"} error was in fact generated by erlang's http packet parsing code. I found the following code in packet_parse_http function from erts/emulator/beam/packet_parser.c file: ... p0 = ptr; while (n && SP(ptr)) { ptr++; n--; } if (ptr==p0) return -1; ... As far as I understand "HTTP/1.1 200\r\n" line does not have any spaces after the status code "200", and the function strips \r\n as a first step of its operation. So the "while" cycle does not run and we get into the "if (ptr==p0) branch" this basically leads to returning of {http_error, "HTTP/1.1 200\r\n"} atom up to the call stack. Strictly speaking this is not a bug in erlang, but I suppose it should take a more relaxed approach to HTTP Status Line parsing and not return http_error if an HTTP response Status Line does not have a Reason-phrase part. From a.zhuravlev@REDACTED Tue Jun 1 23:48:35 2010 From: a.zhuravlev@REDACTED (Alexander Zhuravlev) Date: Wed, 2 Jun 2010 01:48:35 +0400 Subject: Too strict HTTP Status Line parsing In-Reply-To: <20100601212229.GA58321@zaa.local> References: <20100601212229.GA58321@zaa.local> Message-ID: <20100601214835.GA58602@zaa.local> On Wed, Jun 02, 2010 at 01:22:29AM +0400, Alexander Zhuravlev wrote: > Hello, > > I've tried to use lhttpc library (http://bitbucket.org/etc/lhttpc) to fetch > a resource (http://www.qype.com/review/1376848) and got the following > error: > > {{http_error,"HTTP/1.1 200\r\n"}, > [{lhttpc_client,read_response,5}, > {lhttpc_client,execute,8}, > {lhttpc_client,request,9}]} Almost forgot, the error was reproduced with erlang R13B04: Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] lhttpc version 1.2.4 > I've checked lhttpc source code and found out that to receive and parse an HTTP > response it uses _standard_ erlang module gen_tcp on a socket in > {packet, http} mode. So it looks like the {http_error,"HTTP/1.1 200\r\n"} error was > in fact generated by erlang's http packet parsing code. > > I found the following code in packet_parse_http function from > erts/emulator/beam/packet_parser.c file: > > ... > p0 = ptr; > while (n && SP(ptr)) { > ptr++; n--; > } > if (ptr==p0) return -1; > ... > > As far as I understand "HTTP/1.1 200\r\n" line does not have any spaces > after the status code "200", and the function strips \r\n as a first step of > its operation. So the "while" cycle does not run and we get into the > "if (ptr==p0) branch" this basically leads to returning of > {http_error, "HTTP/1.1 200\r\n"} atom up to the call stack. > > Strictly speaking this is not a bug in erlang, but I suppose it > should take a more relaxed approach to HTTP Status Line parsing > and not return http_error if an HTTP response Status Line does not have > a Reason-phrase part. -- Alexander Zhuravlev From sverker@REDACTED Wed Jun 2 12:09:38 2010 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 02 Jun 2010 12:09:38 +0200 Subject: [erlang-bugs] Too strict HTTP Status Line parsing In-Reply-To: <20100601212229.GA58321@zaa.local> References: <20100601212229.GA58321@zaa.local> Message-ID: <4C062DE2.7040008@erix.ericsson.se> Alexander Zhuravlev wrote: > Hello, > > I've tried to use lhttpc library (http://bitbucket.org/etc/lhttpc) to fetch > a resource (http://www.qype.com/review/1376848) and got the following > error: > > {{http_error,"HTTP/1.1 200\r\n"}, > [{lhttpc_client,read_response,5}, > {lhttpc_client,execute,8}, > {lhttpc_client,request,9}]} > > I've checked lhttpc source code and found out that to receive and parse an HTTP > response it uses _standard_ erlang module gen_tcp on a socket in > {packet, http} mode. So it looks like the {http_error,"HTTP/1.1 200\r\n"} error was > in fact generated by erlang's http packet parsing code. > > I found the following code in packet_parse_http function from > erts/emulator/beam/packet_parser.c file: > > ... > p0 = ptr; > while (n && SP(ptr)) { > ptr++; n--; > } > if (ptr==p0) return -1; > ... > > A change to if (ptr==p0 && n>0) return -1; would do it, right? /Sverker, Erlang/OTP > As far as I understand "HTTP/1.1 200\r\n" line does not have any spaces > after the status code "200", and the function strips \r\n as a first step of > its operation. So the "while" cycle does not run and we get into the > "if (ptr==p0) branch" this basically leads to returning of > {http_error, "HTTP/1.1 200\r\n"} atom up to the call stack. > > Strictly speaking this is not a bug in erlang, but I suppose it > should take a more relaxed approach to HTTP Status Line parsing > and not return http_error if an HTTP response Status Line does not have > a Reason-phrase part. > > ________________________________________________________________ > erlang-bugs (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED > > From ingela@REDACTED Wed Jun 2 17:51:48 2010 From: ingela@REDACTED (Ingela Anderton Andin) Date: Wed, 02 Jun 2010 17:51:48 +0200 Subject: [erlang-bugs] SSL 3.11.1 : Certificate-based authentication fails with Firefox In-Reply-To: <81392C6F-3920-4B03-884E-91342E216B18@kallisys.net> References: <81392C6F-3920-4B03-884E-91342E216B18@kallisys.net> Message-ID: <4C067E14.9000802@erix.ericsson.se> Hi ! Thank you for reporting this bug. I have created a fix could you please try it out. git fetch git://github.com/IngelaAndin/otp.git ia/ssl_many_handshake_packages_at_once Regards Ingela Erlang/OTP team, Ericsson AB Paul Guyot wrote: > Hello, > > Certificate-based authentication fails with Firefox (on MacOS X), while the same setup succeeds with Safari, Chrome and curl (which probably use OpenSSL). The server is a very simple erlang SSL server that verifies the peer certificate : > > ssl:ssl_accept(ClientTransportSocket, [{ssl_imp, new}, {active, false}, {fail_if_no_peer_cert, true}, {verify, verify_peer}, {cacertfile, ?ALL_CERTIFICATES}, {certfile, ?SERVER_CERT_FILE}, {keyfile, ?SERVER_KEY_FILE}, {validate_extensions_fun, fun validate_extensions/4}, {verify_fun, fun(ErrorList) -> case ErrorList of [] -> true; _ -> io:format("ErrorList is ~p~n", [ErrorList]), false end end}], infinity) > > The bug seems to be in next_state/3, for the handshake case, around line 1766 of ssl_connection.erl (from the dev branch on github). > > {Packets, Buf} = ssl_handshake:get_tls_handshake(Data,Buf0, KeyAlg,Version), > Start = {next_state, StateName, State0#state{tls_handshake_buffer = Buf}}, > lists:foldl(Handle, Start, Packets) > > Handlers (invoked from Handle) can call next_record to get the next packet. For example, handle_peer_cert/3 (line 1148), is as follows : > > handle_peer_cert(PeerCert, PublicKeyInfo, > #state{session = Session} = State0) -> > State1 = State0#state{session = > Session#session{peer_certificate = PeerCert}, > public_key_info = PublicKeyInfo}, > {Record, State} = next_record(State1), > next_state(certify, Record, State). > > next_record/1 will then attempt to get the next record, ignoring Packets list: > > next_record(#state{tls_cipher_texts = [], socket = Socket} = State) -> > inet:setopts(Socket, [{active,once}]), > {no_record, State}; > next_record(#state{tls_cipher_texts = [CT | Rest], > connection_states = ConnStates0} = State) -> > case ssl_record:decode_cipher_text(CT, ConnStates0) of > {Plain, ConnStates} -> > {Plain, State#state{tls_cipher_texts = Rest, connection_states = ConnStates}}; > #alert{} = Alert -> > {Alert, State} > end. > > If the client sends a single packet in the handshake buffer (i.e. lists:foldl only runs once), everything is fine. The handler will call next_record that will either serve the packet from tls_cipher_texts or handshake with the transport layer with {active, once}. > > If the client sends more than one packet in the handshake buffer (Data + Buf0 above), and more data to the socket that can get returned by next_record, events end up being processed in wrong order. > > This is typically what happens with Firefox. > > Packets is composed of three elements : > > [{{certificate, [<<48,130,5,6,48,...>>]}, <<11, ...>>}, > {{client_key_exchange, {client_diffie_hellman_public, <<...>>}}, <<16,...>>}, > {{certificate_verify, <<120, ...>>}, <<15,...>>}] > > While at the same time, a record is present in tls_cipher_texts. > > Eventually, the process fails with a function_clause error because the state isn't what is expected (in this very case, there is no cipher_state defined here since no client_key_exchange packet was processed): > > ** Reason for termination = > ** {function_clause,[{ssl_cipher,block_decipher, > [#Fun,undefined,20, > <<120,5,167,147,201,226,3,57,134,28,129,147, > 41,178,143,254,181,2,49,7,140,13,191,3,135, > 237,179,175,167,41,239,235,34,22,61,214, > 228,77,124,198,115,53,25,153,151,63,51,197>>, > {3,1}]}, > {ssl_record,decipher,2}, > {ssl_record,decode_cipher_text,2}, > {ssl_connection,next_record,1}, > {ssl_connection,next_state,3}, > {lists,foldl,3}, <--- this is the lists:foldl above > {ssl_connection,next_state,3}, > {gen_fsm,handle_msg,7}]} > > Paul > > > ________________________________________________________________ > erlang-bugs (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED > > > From pguyot@REDACTED Wed Jun 2 18:24:44 2010 From: pguyot@REDACTED (Paul Guyot) Date: Wed, 2 Jun 2010 18:24:44 +0200 Subject: [erlang-bugs] SSL 3.11.1 : Certificate-based authentication fails with Firefox In-Reply-To: <4C067E14.9000802@erix.ericsson.se> References: <81392C6F-3920-4B03-884E-91342E216B18@kallisys.net> <4C067E14.9000802@erix.ericsson.se> Message-ID: <17FF1C3C-9916-475A-8FB7-1AA8D5850EC8@kallisys.net> Hello, Thank you for your quick fix. I confirm that SSL 3.11.1 from your branch works as expected with Firefox and other Gecko-based browsers, as well as with Safari/Chrome/curl. Thanks again ! Regards, Paul Le 2 juin 2010 ? 17:51, Ingela Anderton Andin a ?crit : > Hi ! > > Thank you for reporting this bug. I have created a fix could you please try it out. > > git fetch git://github.com/IngelaAndin/otp.git ia/ssl_many_handshake_packages_at_once > > Regards Ingela Erlang/OTP team, Ericsson AB From a.zhuravlev@REDACTED Wed Jun 2 21:30:45 2010 From: a.zhuravlev@REDACTED (Alexander Zhuravlev) Date: Wed, 2 Jun 2010 23:30:45 +0400 Subject: [erlang-bugs] Too strict HTTP Status Line parsing In-Reply-To: <4C062DE2.7040008@erix.ericsson.se> References: <20100601212229.GA58321@zaa.local> <4C062DE2.7040008@erix.ericsson.se> Message-ID: On Wed, Jun 2, 2010 at 14:09, Sverker Eriksson wrote: > Alexander Zhuravlev wrote: >> >> Hello, >> >> I've tried to use lhttpc library (http://bitbucket.org/etc/lhttpc) to >> fetch >> a resource (http://www.qype.com/review/1376848) and got the following >> error: >> >> {{http_error,"HTTP/1.1 200\r\n"}, >> ? ? ? ?[{lhttpc_client,read_response,5}, >> ? ? ? ?{lhttpc_client,execute,8}, >> ? ? ? ?{lhttpc_client,request,9}]} >> I've checked lhttpc source code and found out that to receive and parse an >> HTTP >> response it uses _standard_ erlang module gen_tcp on a socket in >> {packet, http} mode. So it looks like the {http_error,"HTTP/1.1 200\r\n"} >> error was >> in fact generated by erlang's http packet parsing code. >> >> I found the following code in packet_parse_http function from >> erts/emulator/beam/packet_parser.c file: >> >> ... >> p0 = ptr; >> while (n && SP(ptr)) { >> ? ? ? ?ptr++; n--; >> } >> if (ptr==p0) return -1; >> ... >> >> > > A change to > > if (ptr==p0 && n>0) return -1; > > would do it, right? Yes, probably. But I suppose that fact that the string does not have a phrase string may cause other issues with this call in the packet_parse_http function: return pcb->http_response(arg, major, minor, status, ptr, n); ptr will point to the end of the string and n will be equal to 0. > > /Sverker, Erlang/OTP > >> As far as I understand "HTTP/1.1 200\r\n" line does not have any spaces >> after the status code "200", and the function strips \r\n as a first step >> of >> its operation. So the "while" cycle does not run and we get into the >> "if (ptr==p0) branch" this basically leads to returning of {http_error, >> "HTTP/1.1 200\r\n"} atom up to the call stack. >> >> Strictly speaking this is not a bug in erlang, but I suppose it >> should take a more relaxed approach to HTTP Status Line parsing >> and not return http_error if an HTTP response Status Line does not have >> a Reason-phrase part. >> >> ________________________________________________________________ >> erlang-bugs (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED >> >> > > -- Alexander Zhuravlev From sverker@REDACTED Thu Jun 3 11:38:19 2010 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 03 Jun 2010 11:38:19 +0200 Subject: [erlang-bugs] Too strict HTTP Status Line parsing In-Reply-To: References: <20100601212229.GA58321@zaa.local> <4C062DE2.7040008@erix.ericsson.se> Message-ID: <4C07780B.6090404@erix.ericsson.se> Alexander Zhuravlev wrote: > On Wed, Jun 2, 2010 at 14:09, Sverker Eriksson wrote: > >> Alexander Zhuravlev wrote: >> >>> Hello, >>> >>> I've tried to use lhttpc library (http://bitbucket.org/etc/lhttpc) to >>> fetch >>> a resource (http://www.qype.com/review/1376848) and got the following >>> error: >>> >>> {{http_error,"HTTP/1.1 200\r\n"}, >>> [{lhttpc_client,read_response,5}, >>> {lhttpc_client,execute,8}, >>> {lhttpc_client,request,9}]} >>> I've checked lhttpc source code and found out that to receive and parse an >>> HTTP >>> response it uses _standard_ erlang module gen_tcp on a socket in >>> {packet, http} mode. So it looks like the {http_error,"HTTP/1.1 200\r\n"} >>> error was >>> in fact generated by erlang's http packet parsing code. >>> >>> I found the following code in packet_parse_http function from >>> erts/emulator/beam/packet_parser.c file: >>> >>> ... >>> p0 = ptr; >>> while (n && SP(ptr)) { >>> ptr++; n--; >>> } >>> if (ptr==p0) return -1; >>> ... >>> >>> >>> >> A change to >> >> if (ptr==p0 && n>0) return -1; >> >> would do it, right? >> > > Yes, probably. But I suppose that fact that the string does not have a > phrase string may cause > other issues with this call in the packet_parse_http function: > > return pcb->http_response(arg, major, minor, status, > ptr, n); > > ptr will point to the end of the string and n will be equal to 0 No, that will work. You will get an empty phrase string as part of the http_response-tuple. /Sverker, Erlang/OTP From pguyot@REDACTED Fri Jun 4 00:29:29 2010 From: pguyot@REDACTED (Paul Guyot) Date: Fri, 4 Jun 2010 00:29:29 +0200 Subject: erts_bld_string_n returns negative characters Message-ID: <3E409DAF-0CE0-43A1-9884-85DC6C9F6777@kallisys.net> Hello, I noticed a significant behaviour difference between ssl_imp new and ssl_imp old when using them with {packet, http} due to the fact that ssl_imp old decodes packets through inet driver (and the broker), while ssl_imp new decodes packets with erlang:decode_packet/3 and both do not generate the same data. The (simplified) http packet is: <<71,69,84,32,47,230,157,177,228,186,172,32,72,84,84,80,47,49,46,49,13,10,13,10>> With {ssl_imp old}, I get: {http_request,'GET', {abs_path,[47,230,157,177,228,186,172]}, {1,1}} With {ssl_imp, new}, I get: {http_request,'GET', {abs_path,[47,-26,-99,-79,-28,-70,-84]}, {1,1}} One can get the same result with: erlang:decode_packet(http, <<71,69,84,32,47,230,157,177,228,186,172,32,72,84,84,80,47,49,46,49,13,10,13,10>>, [{packet_size, 0}]). erlang:decode_packet eventually calls erts_bld_string_n. Things happen line 513 of current dev branch on github : http://github.com/erlang/otp/blob/dev/erts/emulator/beam/utils.c#L513 str[i] can be negative (> 0x7f) and therefore promoted to a small negative integer. It seems to me that erts_bld_string_n is supposed to take ISO-8859-1 characters, for example when called from enif_make_string_len (which is therefore broken?). It should return small positive integers instead of negative ones for values > 0x7f. Line 513 should be replaced from: res = CONS(*hpp, make_small(str[i]), res); to: res = CONS(*hpp, make_small((const unsigned char) str[i]), res); or: res = CONS(*hpp, make_small((byte) str[i]), res); This change would mimic what happens with inet_drv. It encodes the string with ERL_DRV_STRING, which is then decoded in beam/io.c with buf_to_intlist, which goes like this: tail = CONS(hp, make_small((byte)*buf), tail); http://github.com/erlang/otp/blob/dev/erts/emulator/beam/utils.c#L2881 Regards, Paul PS: I realize this is not a valid HTTP packet (URIs should be encoded as ASCII 7 bits), but curl 7.20.0 sends it. From pguyot@REDACTED Fri Jun 4 10:22:54 2010 From: pguyot@REDACTED (Paul Guyot) Date: Fri, 4 Jun 2010 10:22:54 +0200 Subject: Bug: erts_bld_string_n returns negative characters In-Reply-To: <3E409DAF-0CE0-43A1-9884-85DC6C9F6777@kallisys.net> References: <3E409DAF-0CE0-43A1-9884-85DC6C9F6777@kallisys.net> Message-ID: Hello, A patch with a test case for enif_make_string is available here: git fetch git://github.com/pguyot/otp.git pg/fix_erts_bld_string_n Regards, Paul Le 4 juin 2010 ? 00:29, Paul Guyot a ?crit : > Hello, > > I noticed a significant behaviour difference between ssl_imp new and ssl_imp old when using them with {packet, http} due to the fact that ssl_imp old decodes packets through inet driver (and the broker), while ssl_imp new decodes packets with erlang:decode_packet/3 and both do not generate the same data. > > The (simplified) http packet is: > <<71,69,84,32,47,230,157,177,228,186,172,32,72,84,84,80,47,49,46,49,13,10,13,10>> > > With {ssl_imp old}, I get: > {http_request,'GET', > {abs_path,[47,230,157,177,228,186,172]}, > {1,1}} > > With {ssl_imp, new}, I get: > {http_request,'GET', > {abs_path,[47,-26,-99,-79,-28,-70,-84]}, > {1,1}} > > One can get the same result with: > erlang:decode_packet(http, <<71,69,84,32,47,230,157,177,228,186,172,32,72,84,84,80,47,49,46,49,13,10,13,10>>, [{packet_size, 0}]). > > erlang:decode_packet eventually calls erts_bld_string_n. Things happen line 513 of current dev branch on github : > > http://github.com/erlang/otp/blob/dev/erts/emulator/beam/utils.c#L513 > > str[i] can be negative (> 0x7f) and therefore promoted to a small negative integer. > > It seems to me that erts_bld_string_n is supposed to take ISO-8859-1 characters, for example when called from enif_make_string_len (which is therefore broken?). It should return small positive integers instead of negative ones for values > 0x7f. Line 513 should be replaced from: > res = CONS(*hpp, make_small(str[i]), res); > to: > res = CONS(*hpp, make_small((const unsigned char) str[i]), res); > or: > res = CONS(*hpp, make_small((byte) str[i]), res); > > This change would mimic what happens with inet_drv. It encodes the string with ERL_DRV_STRING, which is then decoded in beam/io.c with buf_to_intlist, which goes like this: > > tail = CONS(hp, make_small((byte)*buf), tail); > > http://github.com/erlang/otp/blob/dev/erts/emulator/beam/utils.c#L2881 > > Regards, > > Paul > > PS: I realize this is not a valid HTTP packet (URIs should be encoded as ASCII 7 bits), but curl 7.20.0 sends it. > From mikpe@REDACTED Fri Jun 4 11:58:33 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Fri, 4 Jun 2010 11:58:33 +0200 Subject: [erlang-bugs] Bug: erts_bld_string_n returns negative characters In-Reply-To: References: <3E409DAF-0CE0-43A1-9884-85DC6C9F6777@kallisys.net> Message-ID: <19464.52809.200482.73517@pilspetsen.it.uu.se> Paul Guyot writes: > > I noticed a significant behaviour difference between ssl_imp new and ssl_imp old when using them with {packet, http} due to the fact that ssl_imp old decodes packets through inet driver (and the broker), while ssl_imp new decodes packets with erlang:decode_packet/3 and both do not generate the same data. > > > > The (simplified) http packet is: > > <<71,69,84,32,47,230,157,177,228,186,172,32,72,84,84,80,47,49,46,49,13,10,13,10>> > > > > With {ssl_imp old}, I get: > > {http_request,'GET', > > {abs_path,[47,230,157,177,228,186,172]}, > > {1,1}} > > > > With {ssl_imp, new}, I get: > > {http_request,'GET', > > {abs_path,[47,-26,-99,-79,-28,-70,-84]}, > > {1,1}} > > > > One can get the same result with: > > erlang:decode_packet(http, <<71,69,84,32,47,230,157,177,228,186,172,32,72,84,84,80,47,49,46,49,13,10,13,10>>, [{packet_size, 0}]). > > > > erlang:decode_packet eventually calls erts_bld_string_n. Things happen line 513 of current dev branch on github : > > > > http://github.com/erlang/otp/blob/dev/erts/emulator/beam/utils.c#L513 > > > > str[i] can be negative (> 0x7f) and therefore promoted to a small negative integer. > > > > It seems to me that erts_bld_string_n is supposed to take ISO-8859-1 characters, for example when called from enif_make_string_len (which is therefore broken?). It should return small positive integers instead of negative ones for values > 0x7f. Line 513 should be replaced from: > > res = CONS(*hpp, make_small(str[i]), res); > > to: > > res = CONS(*hpp, make_small((const unsigned char) str[i]), res); > > or: > > res = CONS(*hpp, make_small((byte) str[i]), res); > > > > This change would mimic what happens with inet_drv. It encodes the string with ERL_DRV_STRING, which is then decoded in beam/io.c with buf_to_intlist, which goes like this: > > > > tail = CONS(hp, make_small((byte)*buf), tail); I'll just note that the Erlang VM has always been internally inconsistent in how it processes random binary data: sometimes it uses 'unsigned char' pointers (correct), sometimes it uses plain 'char' pointers (incorrect). Earlier I've only seen this result in annoying GCC warnings when building the VM, but here there's a case where the code obviously does the wrong thing at runtime. What's even worse is that the runtime behaviour depends on a subtle property of the underlying machine: whether plain 'char' is signed or unsigned. Casting make_small()'s operand to (byte) or (unsigned char) [no 'const' needed] is the correct short-term fix. From pguyot@REDACTED Sun Jun 6 22:44:57 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 6 Jun 2010 22:44:57 +0200 Subject: Honor start type in rel files when building relup Message-ID: <831A7BD3-52EC-43A1-8A28-F02020CA3780@kallisys.net> Hello, As I mentioned in December 2009 on erlang-bugs [1], there is a bug in sasl where the relup file does not honor the start type from the rel file for newly added applications. A patch with a test case is available here: git fetch git://github.com/pguyot/otp.git pg/honor_start_type_in_rel_files Regards, Paul [1] http://www.erlang.org/cgi-bin/ezmlm-cgi?2:mss:1655:200912:okhiafcddfmanpmaldhm -- Semiocast http://titema.com/ http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From raimo+erlang-bugs@REDACTED Mon Jun 7 09:09:26 2010 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Mon, 7 Jun 2010 09:09:26 +0200 Subject: [erlang-bugs] Honor start type in rel files when building relup In-Reply-To: <831A7BD3-52EC-43A1-8A28-F02020CA3780@kallisys.net> References: <831A7BD3-52EC-43A1-8A28-F02020CA3780@kallisys.net> Message-ID: <20100607070926.GA6238@erix.ericsson.se> On Sun, Jun 06, 2010 at 10:44:57PM +0200, Paul Guyot wrote: > Hello, > > As I mentioned in December 2009 on erlang-bugs [1], there is a bug in sasl where the relup file does not honor the start type from the rel file for newly added applications. > > A patch with a test case is available here: > > git fetch git://github.com/pguyot/otp.git pg/honor_start_type_in_rel_files Thank you! It will be included in 'pu'. The question remains if it will be before or after the release, though... > > Regards, > > Paul > > [1] http://www.erlang.org/cgi-bin/ezmlm-cgi?2:mss:1655:200912:okhiafcddfmanpmaldhm > -- > Semiocast http://titema.com/ > http://semiocast.com/ > +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris > > > ________________________________________________________________ > erlang-bugs (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jonas.boberg@REDACTED Mon Jun 7 14:37:10 2010 From: jonas.boberg@REDACTED (Jonas Boberg) Date: Mon, 7 Jun 2010 12:37:10 +0000 (GMT) Subject: Eunit test process killed when unloading module with tests In-Reply-To: <678672555.439781275914204123.JavaMail.root@zimbra> Message-ID: <1161598867.439801275914230666.JavaMail.root@zimbra> Hi! We've encountered a rare eunit bug related to test generators and code loading. Erlang version: R13B04 (reproduced in R13B03 as well). OS: Mac OS X 10.5, 10.6, Ubuntu 8.1 (probably all platforms). How to reproduce (minimal): * Create two modules: a_mod.erl and b_mod.erl, with the content as seen below * In an Erlang shell: c(a_mod), c(b_mod). eunit:test([a_mod, b_mod]). *unexpected termination of test process* ::killed It seems that after running the a_mod test module, eunit leaves a process that is executing inside it (or that is what we believe). This causes the test process to be killed when the second test runs code:purge(a_mod). * This only occurs when the module that is unloaded has a test generator (first_test_ in this example), not if it has a 'normal' test function. Now, you may wonder why anyone would want to do this. Well, in this minimal example it is not apparent, but we have white-box unit tests in header files that we conditionally include in our modules. Sometimes we mock other modules in the same application, and then we hit this bug. a_mod.erl: ------------------------------------------------- -module(a_mod). -include_lib("eunit/include/eunit.hrl"). first_test_() -> {foreach, fun setup/0, fun teardown/1, [ fun first_/0 ]}. setup() -> ok. teardown(_) -> ok. first_() -> ok. ------------------------------------------------- b_mod.erl: ---------------------- -module(b_mod). -include_lib("eunit/include/eunit.hrl"). second_test() -> code:delete(a_mod), code:purge(a_mod), ok. ---------------------- --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From pguyot@REDACTED Mon Jun 7 18:17:47 2010 From: pguyot@REDACTED (Paul Guyot) Date: Mon, 7 Jun 2010 18:17:47 +0200 Subject: [bug & patch] xmerl_scan doesn't decode &#x refs properly Message-ID: <6F5F6C91-D072-42B4-B102-02E674752F62@kallisys.net> Hello, There is a bug in xmerl_scan. It doesn't decode &#x refs properly. Considering the following code : {UTF8Output, []} = xmerl_scan:string("\n" ++ [229, 145, 156] ++ ""), #xmlElement{content = [#xmlText{value = UTF8Text}]} = UTF8Output, {DecEntityOutput, []} = xmerl_scan:string("\n呜"), #xmlElement{content = [#xmlText{value = DecEntityText}]} = DecEntityOutput, {HexEntityOutput, []} = xmerl_scan:string("\n呜"), #xmlElement{content = [#xmlText{value = HexEntityText}]} = HexEntityOutput, UTF8Text and DecEntityText are equal and as expected ([16#545C]). HexEntityText is (incorrectly) a list composed of the three UTF8 bytes [229, 145, 156] while it should be equal to [16#545C]. A patch with a test case can be found here: git fetch git://github.com/pguyot/otp.git pg/xmerl_scan_hex_entities Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From tims@REDACTED Tue Jun 8 22:09:47 2010 From: tims@REDACTED (Tim Spriggs) Date: Tue, 8 Jun 2010 13:09:47 -0700 Subject: erts configure fails on 64-bit Darwin Message-ID: I am trying to build a 64-bit version of Erlang (R13B04) on Snow Leopard but the erts configure stage fails. The following patch fixes the configure stage: --- configure.in.orig 2010-06-07 13:52:09.000000000 -0700 +++ configure.in 2010-06-07 13:52:56.000000000 -0700 @@ -340,13 +340,13 @@ case $host_os in darwin*) case $CFLAGS in *-m32*) ;; *) - CFLAGS="-m32 $CFLAGS" + # CFLAGS="-m32 $CFLAGS" ;; esac ;; *) if test X${enable_m32_build} = Xyes; then but I am worried that there is some reason that somebody forced 32-bit on Darwin. Any sage advice? TIA, -Tim From tuncer.ayaz@REDACTED Tue Jun 8 23:02:29 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 8 Jun 2010 23:02:29 +0200 Subject: [erlang-bugs] erts configure fails on 64-bit Darwin In-Reply-To: References: Message-ID: On Tue, Jun 8, 2010 at 10:09 PM, Tim Spriggs wrote: > I am trying to build a 64-bit version of Erlang (R13B04) on Snow > Leopard but the erts configure stage fails. The following patch fixes What is the error? What did you pass to ./configure? --enable-darwin-64bit? > the configure stage: > > --- configure.in.orig ? 2010-06-07 13:52:09.000000000 -0700 > +++ configure.in ? ? ? ?2010-06-07 13:52:56.000000000 -0700 > @@ -340,13 +340,13 @@ > ? ? ? case $host_os in > ? ? ? ? ? ? ? darwin*) > ? ? ? ? ? ? ? ? ? ? ? case $CFLAGS in > ? ? ? ? ? ? ? ? ? ? ? *-m32*) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ;; > ? ? ? ? ? ? ? ? ? ? ? *) > - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? CFLAGS="-m32 $CFLAGS" > + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? # CFLAGS="-m32 $CFLAGS" > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ;; > ? ? ? ? ? ? ? ? ? ? ? esac > ? ? ? ? ? ? ? ? ? ? ? ;; > ? ? ? ? ? ? ? *) > ? ? ? ? ? ? ? ? ? ? ? if test X${enable_m32_build} = Xyes; > ? ? ? ? ? ? ? ? ? ? ? then > > > but I am worried that there is some reason that somebody forced 32-bit > on Darwin. Any sage advice? Most likely reason is that wxErlang won't work with wxMac 2.8 (bundled version in Darwin 10.6). Only the unreleased wxWidgets development branch for 2.9 has Cocoa and 64-bit support. Have you tried ./configure with CFLAGS env-var set to "-m32" instead of modifying configure.in? From tims@REDACTED Tue Jun 8 23:51:20 2010 From: tims@REDACTED (Tim Spriggs) Date: Tue, 8 Jun 2010 14:51:20 -0700 Subject: [erlang-bugs] erts configure fails on 64-bit Darwin In-Reply-To: References: Message-ID: On Tue, Jun 8, 2010 at 2:02 PM, Tuncer Ayaz wrote: > On Tue, Jun 8, 2010 at 10:09 PM, Tim Spriggs wrote: >> I am trying to build a 64-bit version of Erlang (R13B04) on Snow >> Leopard but the erts configure stage fails. The following patch fixes > > What is the error? > What did you pass to ./configure? --enable-darwin-64bit? > >> the configure stage: >> >> --- configure.in.orig ? 2010-06-07 13:52:09.000000000 -0700 >> +++ configure.in ? ? ? ?2010-06-07 13:52:56.000000000 -0700 >> @@ -340,13 +340,13 @@ >> ? ? ? case $host_os in >> ? ? ? ? ? ? ? darwin*) >> ? ? ? ? ? ? ? ? ? ? ? case $CFLAGS in >> ? ? ? ? ? ? ? ? ? ? ? *-m32*) >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ;; >> ? ? ? ? ? ? ? ? ? ? ? *) >> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? CFLAGS="-m32 $CFLAGS" >> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? # CFLAGS="-m32 $CFLAGS" >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ;; >> ? ? ? ? ? ? ? ? ? ? ? esac >> ? ? ? ? ? ? ? ? ? ? ? ;; >> ? ? ? ? ? ? ? *) >> ? ? ? ? ? ? ? ? ? ? ? if test X${enable_m32_build} = Xyes; >> ? ? ? ? ? ? ? ? ? ? ? then >> >> >> but I am worried that there is some reason that somebody forced 32-bit >> on Darwin. Any sage advice? > > Most likely reason is that wxErlang won't work with wxMac 2.8 (bundled > version in Darwin 10.6). Only the unreleased wxWidgets development > branch for 2.9 has Cocoa and 64-bit support. > > Have you tried ./configure with CFLAGS env-var set to "-m32" instead > of modifying configure.in? > I am trying to get rid of -m32 since this is a 64-bit build under a 64-bit Gentoo prefix environment. The top level configure command (and a few interesting lines of output): ./configure --prefix=/opt/prefix/usr --build=x86_64-apple-darwin10 --host=x86_64-apple-darwin10 --mandir=/opt/prefix/usr/share/man --infodir=/opt/prefix/usr/share/info --datadir=/opt/prefix/usr/share --sysconfdir=/opt/prefix/etc --localstatedir=/opt/prefix/var/lib --enable-threads --enable-shared-zlib --disable-sctp --disable-hipe --with-ssl=/opt/prefix/usr --enable-dynamic-ssl-lib --enable-kernel-poll --enable-smp-support configure: WARNING: you should use --build, --host, --target configure: WARNING: invalid host type: Ignoring the --cache-file argument since it can cause the system to be erroneously configured Disabling caching checking build system type... x86_64-apple-darwin10 checking host system type... x86_64-apple-darwin10 checking for x86_64-apple-darwin10-gcc... x86_64-apple-darwin10-gcc ... Then later in the configure stage, the erts configure command is called: === configuring in erts (/opt/prefix/var/tmp/portage/dev-lang/erlang-13.2.4/work/otp_src_R13B04/erts) configure: running /bin/sh /opt/prefix/var/tmp/portage/dev-lang/erlang-13.2.4/work/otp_src_R13B04/erts/configure --disable-option-checking '--prefix=/opt/prefix/usr' '--build=x86_64-apple-darwin10' '--host=x86_64-apple-darwin10' '--mandir=/opt/prefix/usr/share/man' '--infodir=/opt/prefix/usr/share/info' '--datadir=/opt/prefix/usr/share' '--sysconfdir=/opt/prefix/etc' '--localstatedir=/opt/prefix/var/lib' '--enable-threads' '--enable-shared-zlib' ' ' '--disable-sctp' '--disable-hipe' '--with-ssl=/opt/prefix/usr' '--enable-dynamic-ssl-lib' '--enable-kernel-poll' '--enable-smp-support' 'build_alias=x86_64-apple-darwin10' 'host_alias=x86_64-apple-darwin10' 'target_alias= ' 'CFLAGS=-O2 -pipe' 'CXXFLAGS=-O2 -pipe' 'LDFLAGS=' --cache-file=/dev/null --srcdir=/opt/prefix/var/tmp/portage/dev-lang/erlang-13.2.4/work/otp_src_R13B04/erts configure: WARNING: you should use --build, --host, --target configure: WARNING: invalid host type: checking build system type... x86_64-apple-darwin10 checking host system type... x86_64-apple-darwin10 The actual error output is: checking whether an emulator with smp support should be built... yes; enabled by user configure: error: cannot build smp enabled emulator since no thread library was found configure: error: /opt/prefix/var/tmp/portage/dev-lang/erlang-13.2.4/work/otp_src_R13B04/erts/configure failed for erts It is failing because there it is calling an x86_64 only version of gcc with -m3. Eg: x86_64-apple-darwin10-gcc -o conftest -m32 -O2 -pipe -I/opt/prefix/var/tmp/portage/dev-lang/erlang-13.2.4/work/otp_src_R13B04/erts/x86_64-apple-darwin10 -no-cpp-precomp -D_XOPEN_SOURCE conftest.c -lutil While there are no enable/disable options set for the bitness in the configure stage, the snippet of code I referenced earlier assumes that -m32 should always be set for Darwin. -Tim From tuncer.ayaz@REDACTED Wed Jun 9 01:03:17 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 9 Jun 2010 01:03:17 +0200 Subject: [erlang-bugs] erts configure fails on 64-bit Darwin In-Reply-To: References: Message-ID: On Tue, Jun 8, 2010 at 11:51 PM, Tim Spriggs wrote: > > While there are no enable/disable options set for the bitness in the > configure stage, the snippet of code I referenced earlier assumes that > -m32 should always be set for Darwin. That has been hard-coded for Snow Leopard as it defaults to -m64 in the toolchain and wxErlang would not work. Can you try --enable-darwin-64bit? From tims@REDACTED Wed Jun 9 01:27:23 2010 From: tims@REDACTED (Tim Spriggs) Date: Tue, 8 Jun 2010 16:27:23 -0700 Subject: [erlang-bugs] erts configure fails on 64-bit Darwin In-Reply-To: References: Message-ID: On Tue, Jun 8, 2010 at 4:03 PM, Tuncer Ayaz wrote: > On Tue, Jun 8, 2010 at 11:51 PM, Tim Spriggs wrote: >> >> While there are no enable/disable options set for the bitness in the >> configure stage, the snippet of code I referenced earlier assumes that >> -m32 should always be set for Darwin. > > That has been hard-coded for Snow Leopard as it defaults to -m64 > in the toolchain and wxErlang would not work. > > Can you try --enable-darwin-64bit? That works and erl starts: Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> 1> q(). ok From tuncer.ayaz@REDACTED Wed Jun 9 01:30:41 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 9 Jun 2010 01:30:41 +0200 Subject: [erlang-bugs] erts configure fails on 64-bit Darwin In-Reply-To: References: Message-ID: On Wed, Jun 9, 2010 at 1:27 AM, Tim Spriggs wrote: > On Tue, Jun 8, 2010 at 4:03 PM, Tuncer Ayaz wrote: >> >> Can you try --enable-darwin-64bit? > > That works and erl starts: > > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [kernel-poll:false] > > Eshell V5.7.5 ?(abort with ^G) > 1> > 1> q(). > ok > Great! You can also try --enable-hipe instead of --disable-hipe. It works in both 32bit and 64bit. From tims@REDACTED Wed Jun 9 02:12:32 2010 From: tims@REDACTED (Tim Spriggs) Date: Tue, 8 Jun 2010 17:12:32 -0700 Subject: [erlang-bugs] erts configure fails on 64-bit Darwin In-Reply-To: References: Message-ID: On Tue, Jun 8, 2010 at 4:30 PM, Tuncer Ayaz wrote: > Great! You can also try --enable-hipe instead of --disable-hipe. > It works in both 32bit and 64bit. Thanks for the tip, I heed the warning in prefix though: if use hipe; then ewarn ewarn "You enabled High performance Erlang. Be aware that this extension" ewarn "can break the compilation in many ways, especially on hardened systems." ewarn "Don't cry, don't file bugs, just disable it! If you have a fix, tell us though on Bugzilla." ewarn fi At this point I'd rather things just worked and worry about performance later :) Once my erlang foo is a little stronger I'll dig into using hipe. Cheers, -Tim PS: x86_64-apple-darwin9 and x86_64-apple-darwin10 both compiled with --enable-darwin-64bit From tuncer.ayaz@REDACTED Wed Jun 9 12:39:18 2010 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 9 Jun 2010 12:39:18 +0200 Subject: [erlang-bugs] erts configure fails on 64-bit Darwin In-Reply-To: References: Message-ID: On Wed, Jun 9, 2010 at 2:12 AM, Tim Spriggs wrote: > PS: x86_64-apple-darwin9 and x86_64-apple-darwin10 both compiled > with --enable-darwin-64bit If darwin9 is 10.5 then yes for the default toolchain environment. No idea about portage on 10.5. From schintke@REDACTED Fri Jun 11 17:26:13 2010 From: schintke@REDACTED (Florian Schintke) Date: Fri, 11 Jun 2010 17:26:13 +0200 Subject: Bug in module random (stdlib-1.16.5, R13B04) Message-ID: <20100611152612.GC11612@csr-pc9.zib.de> Hi, * What steps will reproduce the problem? random:seed({0,0,0}). random:uniform(10000). random:uniform(10000). random:uniform(10000). random:uniform(10000). * What is the expected output? What do you see instead? Expected output are random numbers in the range of 1..10000. Instead all calls to uniform:random(10000) return 1. In general: Randomness of random:uniform is restricted, when a single element of the seed contains a 0 (which regularily is the case when erlang:now() is used for seeding). * What version of the product are you using? On what operating system? stdlib-1.16.5, R13B04 * Please provide any additional information below. stdlib-1.16.5/src/random.erl is supposed to implement the algorithm by B. A. Wichmann and I. D. Hill See "An efficient and portable pseudo-random number generator", Journal of Applied Statistics. AS183. 1982. Also Byte March 1987. In this article each element of the seed has to be a *positive* integer in the range 1..30000. If one element in the seed is 0, this will never change when getting random numbers, as a multiplication with 0 always returns 0. So randomness is broken in this case (see random:uniform()). uniform() -> {A1, A2, A3} = case get(random_seed) of undefined -> seed0(); Tuple -> Tuple end, B1 = (A1*171) rem 30269, B2 = (A2*172) rem 30307, B3 = (A3*170) rem 30323, put(random_seed, {B1,B2,B3}), R = A1/30269 + A2/30307 + A3/30323, R - trunc(R). As seed commonly set using erlang:now(), and at least the third element (milliseconds) of erlang:now() may be 0, the implementation violates the algorithm specification. In general: Randomness of random:uniform is broken, when a single element of the seed contains a 0. * How would you solve this issue? When setting the seed via any random:seed function, 0 should be automatically changed to 1, so erlang:now can savely be used as seed generator. Kind regards, Florian Schintke -- Florian Schintke , http://www.zib.de/schintke/ From magnus@REDACTED Fri Jun 11 18:14:29 2010 From: magnus@REDACTED (Magnus Henoch) Date: Fri, 11 Jun 2010 17:14:29 +0100 Subject: Eunit assert causes Dialyzer warning Message-ID: <841vcdr1ga.fsf@linux-b2a3.site> In R13B04 (and probably others), putting an Eunit ?assert in the code will sometimes cause a Dialyzer warning, as below. This happens because the definition of the assert macro in eunit.hrl does an extra check for when the value is neither true nor false, but in this case Dialyzer can infer that the value must be either true or false. It would be nice if either Eunit or Dialyzer could be changed to not output this warning, so I don't get distracted from warnings that might indicate actual problems. $ cat > foo.erl -module(foo). -include_lib("eunit/include/eunit.hrl"). -export([foo/1]). foo(X) -> ?assert(is_list(X)), X. $ dialyzer --src foo.erl Checking whether the PLT /home/magnus/.dialyzer_plt is up-to-date... yes Proceeding with analysis... foo.erl:8: The variable _ can never match since previous clauses completely covered the type 'false' done in 0m0.57s done (warnings were emitted) -- Magnus Henoch, magnus@REDACTED Erlang Solutions http://www.erlang-solutions.com/ --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From uwe.dauernheim@REDACTED Sat Jun 12 15:56:20 2010 From: uwe.dauernheim@REDACTED (Uwe Dauernheim) Date: Sat, 12 Jun 2010 15:56:20 +0200 Subject: Bug in module random (stdlib-1.16.5, R13B04) Message-ID: Hej Florian, I can only reproduce this result when _all_ three parameters are equal to 0 (Using R13B04). If a single parameter is unequal to 0 the random number generator works as advertised. Getting {0,0,0} by erlang:now() is unlikely, but maybe updating the documentation to reflect the preconditions for random:seed/1 would be a good action point. /Uwe From uwe.dauernheim@REDACTED Sat Jun 12 16:08:56 2010 From: uwe.dauernheim@REDACTED (Uwe Dauernheim) Date: Sat, 12 Jun 2010 16:08:56 +0200 Subject: Bug in module random (stdlib-1.16.5, R13B04) Message-ID: Hej Florian, I have to apologize as I misunderstood your point. Your report is valid. /Uwe From eric.l.2046@REDACTED Sun Jun 13 05:39:52 2010 From: eric.l.2046@REDACTED (Eric Liang) Date: Sun, 13 Jun 2010 11:39:52 +0800 Subject: beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] Message-ID: <4C145308.4080307@gmail.com> Hi all, I got this message from /var/log/messages when I use tsung: May 24 14:55:26 dev-3 kernel: [264002.592197] beam[8449]: segfault at 0 ip 0000000000437e10 sp 00007fffce250948 error 4 in beam[400000+174000] The process is like this one: sunny 4796 0.9 1.8 134320 76444 ? Sl 10:58 0:00 /usr/lib/erlang/erts-5.7.2/bin/beam -A 256 -P 250000 -- -root /usr/lib/erlang -progname erl -- -home /home/sunny -noshell -noinput -noshell -noinput -master [hidden email] -name [hidden email] -s slave slave_start [hidden email] slave_waiter_2 -rsh ssh -noshell -noinput -setcookie tsung -smp disable (Never mind the process number, they're different because the latter is the command when I reproduced the problem.) And the system is: sunny@REDACTED:~$ uname -a Linux dev-3 2.6.31-14-server #48-Ubuntu SMP Fri Oct 16 15:07:34 UTC 2009 x86_64 GNU/Linux sunny@REDACTED:~$ dmesg | grep CPU [ 0.000000] SMP: Allowing 32 CPUs, 24 hotplug CPUs [ 0.000000] NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:32 nr_node_ids:2 [ 0.000000] PERCPU: Remapped at ffffc90000000000 with large pages, static data 90720 bytes [ 0.000000] Initializing CPU#0 [ 0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=32, Nodes=2 [ 0.023346] CPU: Physical Processor ID: 1 [ 0.023347] CPU: Processor Core ID: 0 [ 0.023351] CPU: L1 I cache: 32K, L1 D cache: 32K [ 0.023353] CPU: L2 cache: 256K [ 0.023355] CPU: L3 cache: 4096K [ 0.023358] CPU 0/0x10 -> Node 0 [ 0.023361] mce: CPU supports 9 MCE banks [ 0.023373] CPU0: Thermal monitoring enabled (TM1) [ 0.023377] CPU 0 MCA banks CMCI:2 CMCI:3 CMCI:5 CMCI:6 CMCI:8 [ 0.131977] CPU0: Intel(R) Xeon(R) CPU E5504 @ 2.00GHz stepping 05 I attatched the running process by gdb, and here is the result: Program received signal SIGSEGV, Segmentation fault. unlink_free_block (allctr=0x7ad480, block=0x0) at beam/erl_goodfit_alloc.c:453 453 Uint sz = BLK_SZ(blk); (gdb) whe #0 unlink_free_block (allctr=0x7ad480, block=0x0) at beam/erl_goodfit_alloc.c:453 #1 0x0000000000437fd6 in get_free_block (allctr=0x7ad480, size=, cand_blk=0x0, cand_size=0) at beam/erl_goodfit_alloc.c:421 #2 0x00000000004322c6 in mbc_alloc_block (allctr=0x7ad480, size=72) at beam/erl_alloc_util.c:631 #3 mbc_alloc (allctr=0x7ad480, size=72) at beam/erl_alloc_util.c:758 #4 0x00000000004b1697 in erts_alloc () at beam/erl_alloc.h:179 #5 exit_async () at beam/erl_async.c:132 #6 0x000000000043c13d in system_cleanup (exit_code=) at beam/erl_init.c:1306 #7 0x000000000043c443 in erl_exit (n=0, fmt=0x54649c "") at beam/erl_init.c:1380 #8 0x000000000045d042 in halt_0 (A__p=) at beam/bif.c:3319 #9 0x00000000004d081f in process_main () at beam/beam_emu.c:2008 #10 0x000000000043d56c in erl_start (argc=34, argv=) at beam/erl_init.c:1233 #11 0x00000000004269b9 in main (argc=8049792, argv=0x0) at sys/unix/erl_main.c:29 (gdb) f 1 #1 0x0000000000437fd6 in get_free_block (allctr=0x7ad480, size=, cand_blk=0x0, cand_size=0) at beam/erl_goodfit_alloc.c:421 421 unlink_free_block(allctr, blk); (gdb) l 421 416 /* We are guaranteed to find a block that fits in this bucket */ 417 blk = search_bucket(allctr, min_bi, size); 418 ASSERT(blk); 419 if (cand_blk && cand_size <= BLK_SZ(blk)) 420 return NULL; /* cand_blk was better */ 421 unlink_free_block(allctr, blk); 422 return blk; 423 } 424 425 (gdb) As the running process use the no-debug symbol version beam, I guess the ASSERT in line:418 does not work. So I dig in (gdb) p allctr $1 = (Allctr_t *) 0x7ad480 (gdb) p min_bi $2 = (gdb) p size $3 = (gdb) p *allctr $4 = {name_prefix = 0x534227 "sl_", alloc_no = 3, name = {alloc = 0, realloc = 0, free = 0}, vsn_str = 0x53602f "2.1", t = 0, ramv = 0, sbc_threshold = 524288, sbc_move_threshold = 80, mbc_move_threshold = 50, main_carrier_size = 131072, max_mseg_sbcs = 256, max_mseg_mbcs = 5, largest_mbc_size = 10485760, smallest_mbc_size = 1048576, mbc_growth_stages = 10, mseg_opt = {cache = 1, preserv = 1, abs_shrink_th = 4145152, rel_shrink_th = 80}, mbc_header_size = 32, sbc_header_size = 32, min_mbc_size = 16384, min_mbc_first_free_size = 4096, min_block_size = 32, mbc_list = {first = 0x7f4f93a5d010, last = 0x7f4f93a5d010}, sbc_list = {first = 0x0, last = 0x0}, main_carrier = 0x7f4f93a5d010, get_free_block = 0x437f40 , link_free_block = 0x437d00 , unlink_free_block = 0x437e10 , info_options = 0x438480 , get_next_mbc_size = 0x430e40 , creating_mbc = 0x438100 , destroying_mbc = 0x438100 , init_atoms = 0x4385c0 , mutex = {mtx = {pt_mtx = { __data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' , __align = 0}, is_rec_mtx = 0, prev = 0x0, next = 0x0}}, thread_safe = 0, ts_list = {prev = 0x0, next = 0x0}, atoms_initialized = 0, stopped = 0, calls = {this_alloc = {giga_no = 0, no = 2460}, this_free = {giga_no = 0, no = 2458}, this_realloc = {giga_no = 0, no = 0}, mseg_alloc = {giga_no = 0, no = 0}, mseg_dealloc = {giga_no = 0, no = 0}, mseg_realloc = {giga_no = 0, no = 0}, sys_alloc = {giga_no = 0, no = 1}, sys_free = {giga_no = 0, no = 0}, sys_realloc = {giga_no = 0, no = 0}}, sbcs = {curr_mseg = {no = 0, size = 0}, curr_sys_alloc = {no = 0, size = 0}, max = {no = 0, size = 0}, max_ever = {no = 0, size = 0}, blocks = {curr = {no = 0, size = 0}, max = {no = 0, size = 0}, max_ever = {no = 0, size = 0}}}, mbcs = {curr_mseg = {no = 0, size = 0}, curr_sys_alloc = {no = 1, size = 131112}, max = {no = 1, size = 131112}, max_ever = {no = 0, size = 0}, blocks = {curr = {no = 4, size = 384}, max = {no = 144, size = 13848}, max_ever = {no = 0, size = 0}}}} (gdb) I've discussed this in erlang-questions mailling list with Mikael Pettersson( thanks Mikael ): http://erlang.2086793.n4.nabble.com/beam-8449-segfault-at-0-ip-0000000000437e10-sp-00007fffce250948-error-4-in-beam-400000-174000-td2228295.html#a2244144 It looks like I should ask the question here. Does anyone have any idea on this problem? TIA. Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 261 bytes Desc: OpenPGP digital signature URL: From nem@REDACTED Sun Jun 13 23:02:12 2010 From: nem@REDACTED (Geoff Cant) Date: Sun, 13 Jun 2010 14:02:12 -0700 Subject: Bug: anonymous gen_event process + format_status. Message-ID: Hi all, it seems during the last round of format_status improvements a bug was introduced for anonymous gen_event processes. Repro: {ok, Pid} = gen_event:start(), sys:get_status(Pid). Expected: Status report Actual: Crash due to calling lists:concat on a list containing a pid() I've done some work on format_status here: git fetch git://github.com/archaelus/otp.git format_status_improvements I've fixed the bug by porting the gen_fsm StatusHdr code into gen_event at 00b032608c3ad3573d2b. Then I refactored the StatusHdr code out of gen_server, gen_fsm and gen_event into 'gen' at 9103fa5136d360839aad, and extended the test suite to test for the anon gen_event format status bug at 480f1282d9bcdd7d40c3. Cheers, -- Geoff Cant From raimo+erlang-bugs@REDACTED Mon Jun 14 14:39:01 2010 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Mon, 14 Jun 2010 14:39:01 +0200 Subject: [erlang-bugs] Bug in module random (stdlib-1.16.5, R13B04) In-Reply-To: <20100611152612.GC11612@csr-pc9.zib.de> References: <20100611152612.GC11612@csr-pc9.zib.de> Message-ID: <20100614123901.GA14173@erix.ericsson.se> Thank you for reporting this problem. Your suggested fix to replace 0 with 1 in the seed also seems as the least bad workaround. I am amazed nobody reported this before. It will be fixed or at least documented in earliest feasible release (unfortunately not R14A). On Fri, Jun 11, 2010 at 05:26:13PM +0200, Florian Schintke wrote: > Hi, > > * What steps will reproduce the problem? > > random:seed({0,0,0}). > random:uniform(10000). > random:uniform(10000). > random:uniform(10000). > random:uniform(10000). > > * What is the expected output? What do you see instead? > > Expected output are random numbers in the range of 1..10000. > Instead all calls to uniform:random(10000) return 1. > > In general: Randomness of random:uniform is restricted, when a single > element of the seed contains a 0 (which regularily is the case when > erlang:now() is used for seeding). > > * What version of the product are you using? On what operating system? > > stdlib-1.16.5, R13B04 > > * Please provide any additional information below. > > stdlib-1.16.5/src/random.erl is supposed to implement the algorithm by > B. A. Wichmann and I. D. Hill See "An efficient and portable > pseudo-random number generator", Journal of Applied > Statistics. AS183. 1982. Also Byte March 1987. > > In this article each element of the seed has to be a *positive* > integer in the range 1..30000. > > If one element in the seed is 0, this will never change when getting > random numbers, as a multiplication with 0 always returns 0. So > randomness is broken in this case (see random:uniform()). > > uniform() -> > {A1, A2, A3} = case get(random_seed) of > undefined -> seed0(); > Tuple -> Tuple > end, > B1 = (A1*171) rem 30269, > B2 = (A2*172) rem 30307, > B3 = (A3*170) rem 30323, > put(random_seed, {B1,B2,B3}), > R = A1/30269 + A2/30307 + A3/30323, > R - trunc(R). > > As seed commonly set using erlang:now(), and at least the third > element (milliseconds) of erlang:now() may be 0, the implementation > violates the algorithm specification. > > In general: Randomness of random:uniform is broken, when a single > element of the seed contains a 0. > > * How would you solve this issue? > > When setting the seed via any random:seed function, 0 should be > automatically changed to 1, so erlang:now can savely be used as seed > generator. > > > Kind regards, > > Florian Schintke > -- > Florian Schintke , http://www.zib.de/schintke/ > > ________________________________________________________________ > erlang-bugs (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From schintke@REDACTED Mon Jun 14 23:15:48 2010 From: schintke@REDACTED (Florian Schintke) Date: Mon, 14 Jun 2010 23:15:48 +0200 Subject: [erlang-bugs] Bug in module random (stdlib-1.16.5, R13B04) In-Reply-To: <20100614123901.GA14173@erix.ericsson.se> References: <20100611152612.GC11612@csr-pc9.zib.de> <20100614123901.GA14173@erix.ericsson.se> Message-ID: <20100614211548.GA12683@csr-pc9.fritz.box> [Raimo Niskanen] > Thank you for reporting this problem. Your suggested fix to replace > 0 with 1 in the seed also seems as the least bad workaround. > > I am amazed nobody reported this before. > > It will be fixed or at least documented in earliest feasible release > (unfortunately not R14A). To also fix the additional problem that a 0 could be generated by the modulo operation when numbers for a seed are larger than the prime numbers used in the algorithm, one should do first the calculation as currently done and then check whether the result is zero. So my proposal for a new random:seed/3 would be something similar to this: seed(A1, A2, A3) -> case abs(A1) rem 30269 of 0 -> T1 = 1; T1 -> T1 end, case abs(A2) rem 30307 of 0 -> T2 = 1; T2 -> T2 end, case abs(A3) rem 30323 of 0 -> T3 = 1; T3 -> T3 end, put(random_seed, {T1, T2, T3}). Florian -- Florian Schintke , http://www.zib.de/schintke/ From raimo+erlang-bugs@REDACTED Tue Jun 15 09:30:41 2010 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Tue, 15 Jun 2010 09:30:41 +0200 Subject: [erlang-bugs] Bug in module random (stdlib-1.16.5, R13B04) In-Reply-To: <20100614211548.GA12683@csr-pc9.fritz.box> References: <20100611152612.GC11612@csr-pc9.zib.de> <20100614123901.GA14173@erix.ericsson.se> <20100614211548.GA12683@csr-pc9.fritz.box> Message-ID: <20100615073041.GA32448@erix.ericsson.se> On Mon, Jun 14, 2010 at 11:15:48PM +0200, Florian Schintke wrote: > [Raimo Niskanen] > > Thank you for reporting this problem. Your suggested fix to replace > > 0 with 1 in the seed also seems as the least bad workaround. > > > > I am amazed nobody reported this before. > > > > It will be fixed or at least documented in earliest feasible release > > (unfortunately not R14A). > > To also fix the additional problem that a 0 could be generated by the > modulo operation when numbers for a seed are larger than the prime > numbers used in the algorithm, one should do first the calculation as > currently done and then check whether the result is zero. So my > proposal for a new random:seed/3 would be something similar to this: > > seed(A1, A2, A3) -> > case abs(A1) rem 30269 of 0 -> T1 = 1; T1 -> T1 end, > case abs(A2) rem 30307 of 0 -> T2 = 1; T2 -> T2 end, > case abs(A3) rem 30323 of 0 -> T3 = 1; T3 -> T3 end, > put(random_seed, {T1, T2, T3}). Great! Well spotted again. Something like that... > > > Florian > -- > Florian Schintke , http://www.zib.de/schintke/ > > ________________________________________________________________ > erlang-bugs (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From dougedmunds@REDACTED Wed Jun 16 19:54:36 2010 From: dougedmunds@REDACTED (Doug Edmunds (gmail)) Date: Wed, 16 Jun 2010 10:54:36 -0700 Subject: wxErlang missing a module Message-ID: <4C190FDC.9030100@gmail.com> wxErlang 0.98.5 is missing a module for wxSystemSettings. "wxSystemSettings allows the application to ask for details about the system. This can include settings such as standard colours, fonts, and user interface element sizes." see: http://docs.wxwidgets.org/stable/wx_wxsystemsettings.html#wxsystemsettings --dae From dougedmunds@REDACTED Sat Jun 19 09:05:45 2010 From: dougedmunds@REDACTED (Doug Edmunds (gmail)) Date: Sat, 19 Jun 2010 00:05:45 -0700 Subject: wxDC documentation of drawArc/6 should be drawArc/4 Message-ID: <4C1C6C49.3030300@gmail.com> The left-column of the docs for wxDC have an entry for drawArc/6. It should be drawArc/4. Link http://www.erlang.org/doc/man/wxDC.html#drawArc-6 should be http://www.erlang.org/doc/man/wxDC.html#drawArc-4 The right column is correct: drawArc(This::wxDC(), Pt1::{X::integer(), Y::integer()}, Pt2::{X::integer(), Y::integer()}, Centre::{X::integer(), Y::integer()}) -> ok From jonas.boberg@REDACTED Mon Jun 21 16:43:30 2010 From: jonas.boberg@REDACTED (Jonas Boberg) Date: Mon, 21 Jun 2010 14:43:30 +0000 (GMT) Subject: gen_tcp/new_ssl http packet inconsistency when reading malformed header In-Reply-To: <1796551679.502961277131072505.JavaMail.root@zimbra> Message-ID: <177978842.502991277131410123.JavaMail.root@zimbra> The new_ssl and gen_tcp sockets does not return http packet decode errors in the same way. How to repeat: 1. Run a ssl(new_ssl)/gen_tcp server with the http packet option set 2. Send a http request with a malformed header field to the server new_ssl/gen_tcp will return different values when reading the header: ssl:recv(SSLSocket, 0) -> {ok, {http_error, _Str}} gen_tcp:recv(Socket, 0) -> {error, {http_error, _Str}} Erlang version: R13B04 (no fix mentioned in in the release notes of R14A). /Jonas --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From ingela@REDACTED Tue Jun 22 11:04:07 2010 From: ingela@REDACTED (Ingela Anderton Andin) Date: Tue, 22 Jun 2010 11:04:07 +0200 Subject: [erlang-bugs] gen_tcp/new_ssl http packet inconsistency when reading malformed header In-Reply-To: <177978842.502991277131410123.JavaMail.root@zimbra> References: <177978842.502991277131410123.JavaMail.root@zimbra> Message-ID: <4C207C87.7090905@erix.ericsson.se> Hi! Thank you for reporting this, we will change ssl to work the same way as gen_tcp. Regards Ingela Erlang/OTP team - Ericsson AB Jonas Boberg wrote: > The new_ssl and gen_tcp sockets does not return > http packet decode errors in the same way. > > How to repeat: > 1. Run a ssl(new_ssl)/gen_tcp server with the http packet option set > 2. Send a http request with a malformed header field to the server > > new_ssl/gen_tcp will return different values when > reading the header: > ssl:recv(SSLSocket, 0) -> {ok, {http_error, _Str}} > gen_tcp:recv(Socket, 0) -> {error, {http_error, _Str}} > > Erlang version: R13B04 (no fix mentioned in in the release notes of R14A). > > /Jonas > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-bugs (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED > > From ulf.norell@REDACTED Tue Jun 22 11:41:09 2010 From: ulf.norell@REDACTED (Ulf Norell) Date: Tue, 22 Jun 2010 11:41:09 +0200 Subject: fun Name/Arity syntax doesn't work for imported functions Message-ID: The "fun Name/Arity" syntax doesn't take imports into account, requiring imported functions to be qualified with their module names. Example: --- -module(bug). -compile(export_all). -import(lists, [sort/1]). ok(Xss) -> lists:map(fun(Xs) -> sort(Xs) end, Xss). ok2(Xss) -> lists:map(fun lists:sort/1, Xss). bad(Xss) -> lists:map(fun sort/1, Xss). --- 1> c(bug). ./bug.erl:13: function sort/1 undefined / Ulf From ingela@REDACTED Tue Jun 22 15:28:20 2010 From: ingela@REDACTED (Ingela Anderton Andin) Date: Tue, 22 Jun 2010 15:28:20 +0200 Subject: [erlang-bugs] gen_tcp/new_ssl http packet inconsistency when reading malformed header In-Reply-To: <4C207C87.7090905@erix.ericsson.se> References: <177978842.502991277131410123.JavaMail.root@zimbra> <4C207C87.7090905@erix.ericsson.se> Message-ID: <4C20BA74.7090709@erix.ericsson.se> Hi again! Maybe I spoke to early. I according to the documentation: "A socket in passive mode will return {ok, HttpPacket} from gen_tcp:recv" "HttpPacket = HttpRequest | HttpResponse | HttpHeader | http_eoh | HttpError" "HttpError = {http_error, HttpString}" So ssl does acctualy return what the documentation says it should. I am not sure if we dare change the gen_tcp:recv or not but that would be the "right" thing to do. Regards Ingela Erlang/OTP team - Ericsson AB Ingela Anderton Andin wrote: > Hi! > > Thank you for reporting this, we will change ssl to work the same way > as gen_tcp. > > Regards Ingela Erlang/OTP team - Ericsson AB > > Jonas Boberg wrote: >> The new_ssl and gen_tcp sockets does not return >> http packet decode errors in the same way. >> >> How to repeat: >> 1. Run a ssl(new_ssl)/gen_tcp server with the http packet option set >> 2. Send a http request with a malformed header field to the server >> >> new_ssl/gen_tcp will return different values when reading the header: >> ssl:recv(SSLSocket, 0) -> {ok, {http_error, _Str}} >> gen_tcp:recv(Socket, 0) -> {error, {http_error, _Str}} >> Erlang version: R13B04 (no fix mentioned in in the release notes of >> R14A). >> >> /Jonas >> --------------------------------------------------- >> >> --------------------------------------------------- >> >> WE'VE CHANGED NAMES! >> >> Since January 1st 2010 Erlang Training and Consulting Ltd. has become >> ERLANG SOLUTIONS LTD. >> >> www.erlang-solutions.com >> >> >> ________________________________________________________________ >> erlang-bugs (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED >> >> > > > ________________________________________________________________ > erlang-bugs (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED > From matthias@REDACTED Wed Jun 23 01:09:13 2010 From: matthias@REDACTED (Matthias Lang) Date: Wed, 23 Jun 2010 01:09:13 +0200 Subject: otp-build does not run as expected if called from a Makefile Message-ID: <20100622230913.GA2296@corelatus.se> Hi, in R13B04 (it looks like R14A is the same, but I haven't tried), the top-level script 'otp_build' doesn't work as expected if it's called from a Makefile. Here's an example of it going wrong in R13B04 (these commands are being issued by make, not at the command line): ./otp_build configure --xcomp-conf=xcomp/erl-xcomp-mips-linux.conf sh -c "cd otp_src_R13B04; ./otp_build boot -a" [: 1432: Xmake[1]:: unexpected operator make[1]: Entering directory `/home/matthias/corelatus/rootfs/packages/erlang/otp_src_R13B04' * * Cross compiling Erlang/OTP-R13B04 for: x86_64-unknown-linux-gnu * ... otp_build then works for quite some time (minutes) before running into trouble. The final error message has nothing to do with the actual problem, the clue of the real problem is the "unexpected operator" line above. The problem is that otp_build has a few lines like this: if [ X`$MAKE is_cross_configured` = Xyes ]; then usually, if you run that line, you get the answer "yes". But if otp_build was called by make, then the make invoked by the line above detects that it's running inside another make and the answer becomes "make[1]: Entering directory `/home/matthias/corelatus/rootfs/packages/erlang/otp_src_R13B04'yes" Solutions I can think of: 1. Don't allow running otp_build from within make, since it does the wrong thing. Fail early. I've appended a patch for that below (against R14A) 2. use make's --no-print-directory switch to suppress the extra junk. That's a nice solution which actually solves the problem, but I'm not sure it'll work for clearmake (and BSD?), though. 3. conceal the fact that make is running, by clearing MAKELEVEL and MAKEFLAGS. That seems dirty. Matt ====================================================================== diff --git a/otp_build b/otp_build index 1744351..33424d8 100755 --- a/otp_build +++ b/otp_build @@ -870,6 +870,11 @@ lookup_prog_in_path () setup_make () { + if [ X$MAKELEVEL != X ]; then + echo "This script works badly if run by 'make'. Aborting." + exit 1 + fi + if [ -z "$MAKE" ]; then case $TARGET in win32) From jonas.boberg@REDACTED Wed Jun 23 11:04:40 2010 From: jonas.boberg@REDACTED (Jonas Boberg) Date: Wed, 23 Jun 2010 09:04:40 +0000 (GMT) Subject: badarg in new_ssl ssl_connection with client socket and {packet, http} In-Reply-To: <833538408.512451277283321691.JavaMail.root@zimbra> Message-ID: <1058005558.512631277283880461.JavaMail.root@zimbra> Hi! The http packet option does not work with new_ssl client sockets. The ssl_connection process crashes with a badarg when retrieving the http response: ::exit:{{badarg,[{erlang,binary_to_list,[{http_response,{1,1},200,"OK"}]}, {ssl_connection,format_reply,3}, {ssl_connection,format_reply,2}, {ssl_connection,deliver_app_data,4}, {ssl_connection,application_data,2}, {ssl_connection,handle_event,3}, {gen_fsm,handle_msg,7}, {proc_lib,init_p_do_apply,3}]}, {gen_fsm,sync_send_all_state_event,[<0.592.0>,{recv,0},1000]}} in function gen_fsm:sync_send_all_state_event/3 in call from ssl_connection:sync_send_all_state_event/3 Erlang version: R13B04 (no fix mentioned in in the release notes of R14A). How to repeat: * ssl:connect to a http server * ssl:send a http request string * call ssl:recv on the socket Trace that shows the calls: (<0.592.0>) call ssl:connect("localhost",10000,[{ssl_imp,new},{packet,http},{active,false}]) (<0.592.0>) returned from ssl:connect/3 -> {ok,{sslsocket,new_ssl,<0.594.0>}} (<0.592.0>) call ssl:send({sslsocket,new_ssl,<0.594.0>},"GET / HTTP/1.1\r\n\r\n") (<0.592.0>) returned from ssl:send/2 -> ok (<0.592.0>) call ssl:recv({sslsocket,new_ssl,<0.594.0>},0,1000) (<0.592.0>) exception_from {ssl,recv,3} {exit,{{badarg,[{erlang,binary_to_list,[{http_response,{1,1},200,"OK"}]}, {ssl_connection,format_reply,3}, {ssl_connection,format_reply,2}, {ssl_connection,deliver_app_data,4}, {ssl_connection,application_data,2}, {ssl_connection,handle_event,3}, {gen_fsm,handle_msg,7}, {proc_lib,init_p_do_apply,3}]}, {gen_fsm,sync_send_all_state_event,[<0.594.0>,{recv,0},1000]}}} /Jonas --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From jonas.boberg@REDACTED Wed Jun 23 11:14:11 2010 From: jonas.boberg@REDACTED (Jonas Boberg) Date: Wed, 23 Jun 2010 09:14:11 +0000 (GMT) Subject: [erlang-bugs] gen_tcp/new_ssl http packet inconsistency when reading malformed header In-Reply-To: <4C20BA74.7090709@erix.ericsson.se> Message-ID: <1807144371.512761277284451607.JavaMail.root@zimbra> Ingela Anderton Andin wrote: > Maybe I spoke to early. I according to the documentation: > "A socket in passive mode will return {ok, HttpPacket} from gen_tcp:recv" > ... > So ssl does acctualy return what the documentation says it should. Yes, actually dialyzer even gives a warning for this code in R13B04: case gen_tcp:recv(Socket, 0, 1000) of %% dialyzer gives a warning for the following line, but its "correct" in R13B {error, {http_error, _Str}} -> ... end If it's not possible to change the gen_tcp behaviour, I guess the documentation and type specs should be updated. /Jonas --------------------------------------------------- --------------------------------------------------- WE'VE CHANGED NAMES! Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. www.erlang-solutions.com From alex.demidenko@REDACTED Wed Jun 23 11:43:27 2010 From: alex.demidenko@REDACTED (Alexander Demidenko) Date: Wed, 23 Jun 2010 16:43:27 +0700 Subject: [bug] erl_interface-3.6.5: C library erl_marshal, function erl_decode Message-ID: Hi! I found incorrect behavior of the function. Easiest way to demonstrate in the following example: dummy C code: * * * byte * bb = new byte[10000]; ETERM* t = erl_mk_int(-1250947744); erl_encode(t, bb); ETERM* d = erl_decode(bb); DBGTERM("Result: ", d);* Output: *12:52:11 : Result: -1250947745* * * Some strange in function decode_long, while converting long to ulong: *long n = -((long)u);* instead: *not R, inc R,* compiler generate: *not R, or R, 1* *I suspect that the problem in compiler optimization options.* I found same bug in binary erl_interface by: Erlang R13B01 (erts-5.7.2) from standart ubuntu lucid repository. Erlang R13B04 (erts-5.7.5) from deb http://ppa.launchpad.net/erlang-dev/ppa/ubuntu karmic main -- --------------------------------------------- With best regards, Alexander. From ingela@REDACTED Wed Jun 23 15:41:52 2010 From: ingela@REDACTED (Ingela Anderton Andin) Date: Wed, 23 Jun 2010 15:41:52 +0200 Subject: [erlang-bugs] badarg in new_ssl ssl_connection with client socket and {packet, http} In-Reply-To: <1058005558.512631277283880461.JavaMail.root@zimbra> References: <1058005558.512631277283880461.JavaMail.root@zimbra> Message-ID: <4C220F20.4090601@erix.ericsson.se> Hi! This was fixed in ssl-3.11.1. From release notes: (Second part is the relevant one). "Alert handling has been improved to better handle unexpected but valid messages and the implementation is also changed to avoid timing related issues that could cause different error messages depending on network latency. Packet handling was sort of broken but would mostly work as expected when socket was in binary mode. This has now been fixed. Own Id: OTP-8588" Although the patch ssl-3.11.1 on R13B has not been released as no one requested it. We just want to have a ssl version on R13B that is as close as possible to ssl-4.0 (if someone needs it) . Regards Ingela Erlang/OTP team - Ericsson AB Jonas Boberg wrote: > Hi! > > The http packet option does not work with new_ssl client sockets. > The ssl_connection process crashes with a badarg when retrieving the http response: > ::exit:{{badarg,[{erlang,binary_to_list,[{http_response,{1,1},200,"OK"}]}, > {ssl_connection,format_reply,3}, > {ssl_connection,format_reply,2}, > {ssl_connection,deliver_app_data,4}, > {ssl_connection,application_data,2}, > {ssl_connection,handle_event,3}, > {gen_fsm,handle_msg,7}, > {proc_lib,init_p_do_apply,3}]}, > {gen_fsm,sync_send_all_state_event,[<0.592.0>,{recv,0},1000]}} > in function gen_fsm:sync_send_all_state_event/3 > in call from ssl_connection:sync_send_all_state_event/3 > > Erlang version: R13B04 (no fix mentioned in in the release notes of R14A). > > How to repeat: > * ssl:connect to a http server > * ssl:send a http request string > * call ssl:recv on the socket > > Trace that shows the calls: > (<0.592.0>) call ssl:connect("localhost",10000,[{ssl_imp,new},{packet,http},{active,false}]) > (<0.592.0>) returned from ssl:connect/3 -> {ok,{sslsocket,new_ssl,<0.594.0>}} > > (<0.592.0>) call ssl:send({sslsocket,new_ssl,<0.594.0>},"GET / HTTP/1.1\r\n\r\n") > (<0.592.0>) returned from ssl:send/2 -> ok > > (<0.592.0>) call ssl:recv({sslsocket,new_ssl,<0.594.0>},0,1000) > > (<0.592.0>) exception_from {ssl,recv,3} {exit,{{badarg,[{erlang,binary_to_list,[{http_response,{1,1},200,"OK"}]}, > {ssl_connection,format_reply,3}, > {ssl_connection,format_reply,2}, > {ssl_connection,deliver_app_data,4}, > {ssl_connection,application_data,2}, > {ssl_connection,handle_event,3}, > {gen_fsm,handle_msg,7}, > {proc_lib,init_p_do_apply,3}]}, > {gen_fsm,sync_send_all_state_event,[<0.594.0>,{recv,0},1000]}}} > > > /Jonas > --------------------------------------------------- > > --------------------------------------------------- > > WE'VE CHANGED NAMES! > > Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD. > > www.erlang-solutions.com > > > ________________________________________________________________ > erlang-bugs (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED > > From anders.nygren@REDACTED Fri Jun 25 22:19:14 2010 From: anders.nygren@REDACTED (Anders Nygren) Date: Fri, 25 Jun 2010 15:19:14 -0500 Subject: [erlang-bugs] mnesia disc_copies memory size corrupts w/HiPE 64-bit In-Reply-To: <4A8C9078.2080700@alertlogic.net> References: <4A8C9078.2080700@alertlogic.net> Message-ID: Hi Was there any resolution to this? We are running into the same thing on Solaris 10 x86 with OTP13B02-1 32 bit $gcc -v Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.10/3.4.3/specs Configured with: /builds/sfw10-gate/usr/src/cmd/gcc/gcc-3.4.3/configure --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++ --enable-shared Thread model: posix gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) Any chance that upgrading to R13B04 would help? /Anders On Wed, Aug 19, 2009 at 6:53 PM, Paul Fisher wrote: > I have a mnesia table that is defined as disc_copies, which ends up with a > corrupt memory size value after being updated from concurrent (erlang) > processes from code compiled with HiPE. ?All updates to the table are done > as mnesia:transaction/1 calls. > > If I update the table from only two processes the memory size is correct: > > (emacs@REDACTED)2> mnesia:table_info(clu_unassignedq,size). > 0 > (emacs@REDACTED)3> mnesia:table_info(clu_unassignedq,memory). > 90 > > After updating the table via 11 processes, the table size ends up zero, but > memory size is completely bogus: > > (emacs@REDACTED)10> mnesia:table_info(clu_unassignedq,size). > 0 > (emacs@REDACTED)11> mnesia:table_info(clu_unassignedq,memory). > 2305843009213693459 > > If the same thing is done with the code calling the mnesia operations is > *not* compiled with HiPE, all is well and everything appears correct. > > This appears to be an artifact of the SMP/ets concurrency improvements > introduced in R13. Thoughts for further investigation? > > Platform is amd64 (intel core 2) on opensolaris. ?Erlang R13B02 snapshot > from 2009/08/12. > > pfisher@REDACTED:~/lm/third_party/erlang-R13/install/solaris$ gcc -v > Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.11/3.4.3/specs > Configured with: /builds2/sfwnv-gate/usr/src/cmd/gcc/gcc-3.4.3/configure > --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as > --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++,f77,objc > --enable-shared > Thread model: posix > gcc version 3.4.3 (csl-sol210-3_4-20050802) > > > -- > paul > > ________________________________________________________________ > erlang-bugs mailing list. See http://www.erlang.org/faq.html > erlang-bugs (at) erlang.org > > From anders.nygren@REDACTED Fri Jun 25 23:54:38 2010 From: anders.nygren@REDACTED (Anders Nygren) Date: Fri, 25 Jun 2010 16:54:38 -0500 Subject: [erlang-bugs] mnesia disc_copies memory size corrupts w/HiPE 64-bit In-Reply-To: References: <4A8C9078.2080700@alertlogic.net> Message-ID: On Fri, Jun 25, 2010 at 3:19 PM, Anders Nygren wrote: > Hi > Was there any resolution to this? Newermind, I found it, OTP-8377 /Anders > We are running into the same thing on Solaris 10 x86 with OTP13B02-1 > 32 bit > > $gcc -v > Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.10/3.4.3/specs > Configured with: > /builds/sfw10-gate/usr/src/cmd/gcc/gcc-3.4.3/configure > --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as > --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++ > --enable-shared > Thread model: posix > gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) > > Any chance that upgrading to R13B04 would help? > > /Anders > > On Wed, Aug 19, 2009 at 6:53 PM, Paul Fisher wrote: >> I have a mnesia table that is defined as disc_copies, which ends up with a >> corrupt memory size value after being updated from concurrent (erlang) >> processes from code compiled with HiPE. ?All updates to the table are done >> as mnesia:transaction/1 calls. >> >> If I update the table from only two processes the memory size is correct: >> >> (emacs@REDACTED)2> mnesia:table_info(clu_unassignedq,size). >> 0 >> (emacs@REDACTED)3> mnesia:table_info(clu_unassignedq,memory). >> 90 >> >> After updating the table via 11 processes, the table size ends up zero, but >> memory size is completely bogus: >> >> (emacs@REDACTED)10> mnesia:table_info(clu_unassignedq,size). >> 0 >> (emacs@REDACTED)11> mnesia:table_info(clu_unassignedq,memory). >> 2305843009213693459 >> >> If the same thing is done with the code calling the mnesia operations is >> *not* compiled with HiPE, all is well and everything appears correct. >> >> This appears to be an artifact of the SMP/ets concurrency improvements >> introduced in R13. Thoughts for further investigation? >> >> Platform is amd64 (intel core 2) on opensolaris. ?Erlang R13B02 snapshot >> from 2009/08/12. >> >> pfisher@REDACTED:~/lm/third_party/erlang-R13/install/solaris$ gcc -v >> Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.11/3.4.3/specs >> Configured with: /builds2/sfwnv-gate/usr/src/cmd/gcc/gcc-3.4.3/configure >> --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as >> --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++,f77,objc >> --enable-shared >> Thread model: posix >> gcc version 3.4.3 (csl-sol210-3_4-20050802) >> >> >> -- >> paul >> >> ________________________________________________________________ >> erlang-bugs mailing list. See http://www.erlang.org/faq.html >> erlang-bugs (at) erlang.org >> >> > From pguyot@REDACTED Sat Jun 26 20:05:41 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sat, 26 Jun 2010 20:05:41 +0200 Subject: HiPE crash with crypto and R14A Message-ID: <21063A7B-55E0-466C-81AE-31E8F7CFA4AB@kallisys.net> Hello, I noticed the following reproduceable crash (segment fault) on R14A with HiPE and crypto 2.0 on both MacOS X 64bits build and FreeBSD 64bits build. It may be related to the fact that crypto now uses NIF. hipe_crash.erl: ---- -module(hipe_crash). -export([crash/0]). crash() -> ok = application:start(crypto), Stage1 = crypto:sha("test"), crypto:sha(Stage1). ---- erlc +native hipe_crash.erl erl -eval 'hipe_crash:crash()' Compiling without +native does not yield a crash. crypto:sha("test") is <<169,74,143,229,204,177,155,166,28,76,8,115,211,145,233,135,152,47,187,211>>, yet the following code does not crash : ok = application:start(crypto), crypto:sha(<<169,74,143,229,204,177,155,166,28,76,8,115,211,145,233,135,152,47,187,211>>). Regards, Paul From paul.joseph.davis@REDACTED Sat Jun 26 20:37:57 2010 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Sat, 26 Jun 2010 14:37:57 -0400 Subject: [erlang-bugs] HiPE crash with crypto and R14A In-Reply-To: <21063A7B-55E0-466C-81AE-31E8F7CFA4AB@kallisys.net> References: <21063A7B-55E0-466C-81AE-31E8F7CFA4AB@kallisys.net> Message-ID: On Sat, Jun 26, 2010 at 2:05 PM, Paul Guyot wrote: > Hello, > > I noticed the following reproduceable crash (segment fault) on R14A with HiPE and crypto 2.0 on both MacOS X 64bits build and FreeBSD 64bits build. It may be related to the fact that crypto now uses NIF. > > hipe_crash.erl: > ---- > -module(hipe_crash). > -export([crash/0]). > > crash() -> > ? ?ok = application:start(crypto), > ? ?Stage1 = crypto:sha("test"), > ? ?crypto:sha(Stage1). > ---- > erlc +native hipe_crash.erl > erl -eval 'hipe_crash:crash()' > > Compiling without +native does not yield a crash. > > crypto:sha("test") is <<169,74,143,229,204,177,155,166,28,76,8,115,211,145,233,135,152,47,187,211>>, yet the following code does not crash : > > ? ?ok = application:start(crypto), > ? ?crypto:sha(<<169,74,143,229,204,177,155,166,28,76,8,115,211,145,233,135,152,47,187,211>>). > > Regards, > > Paul > > > ________________________________________________________________ > erlang-bugs (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED > > Confirmed on Ubuntu 10.04 with R14A from source. I grabbed a backtrace as well but it isn't very informative. Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xb6484b70 (LWP 23582)] 0x00000000 in ?? () (gdb) bt #0 0x00000000 in ?? () #1 0xb763b5a4 in ?? () #2 0xb74e82c4 in ?? () Backtrace stopped: previous frame inner to this frame (corrupt stack?) HTH, Paul Davis From pguyot@REDACTED Sun Jun 27 01:09:04 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 27 Jun 2010 01:09:04 +0200 Subject: [erlang-bugs] HiPE crash with crypto and R14A In-Reply-To: References: <21063A7B-55E0-466C-81AE-31E8F7CFA4AB@kallisys.net> Message-ID: <79445742-14F4-4087-BADF-1CD38450D98A@kallisys.net> Le 26 juin 2010 ? 20:37, Paul Davis a ?crit : > Confirmed on Ubuntu 10.04 with R14A from source. I grabbed a backtrace > as well but it isn't very informative. > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0xb6484b70 (LWP 23582)] > 0x00000000 in ?? () > (gdb) bt > #0 0x00000000 in ?? () > #1 0xb763b5a4 in ?? () > #2 0xb74e82c4 in ?? () > Backtrace stopped: previous frame inner to this frame (corrupt stack?) The crash is not (directly) caused by a NIF but by on_load used on crypto (to load the NIFs). Here is a simpler test case: ---- dummy_with_on_load.erl ---- -module(dummy_with_on_load). -on_load(on_load/0). -export([foo/0]). foo() -> <<"hello">>. on_load() -> ok. ---- ---- hipe_crash.erl ---- -module(hipe_crash). -export([crash/0]). crash() -> dummy_with_on_load:foo(), dummy_with_on_load:foo(). ---- erlc dummy_with_on_load.erl erlc +native hipe_crash.erl erl -eval 'hipe_crash:crash()' Notice the lack of +native for dummy_with_on_load.erl, hence it's not (exactly) what the following workaround is about: http://github.com/erlang/otp/commit/3befa76d1e24108acb5cb757157c2fb3384a8e51 Yet, it might be the same bug.... The crash can be avoided by commenting the following line: /* * The on_load function succeded. Fix up export entries. */ for (i = 0; i < export_list_size(); i++) { Export *ep = export_list(i); if (ep != NULL && ep->code[0] == BIF_ARG_1 && ep->code[4] != 0) { ep->address = (void *) ep->code[4]; ep->code[3] = 0; <----- ep->code[4] = 0; } } http://github.com/erlang/otp/blob/dev/erts/emulator/beam/beam_bif_load.c#L340 I just do not understand yet why resetting code[3] to 0 fixes up the export entry. The emulator has a copy of the value of ep->code[3] for the exported function (foo in this test case, sha in the previous test case) before on_load is called. on_load is called, ep->code[3] is reset. The first call succeeds nevertheless because there is a copy. However, the second call fails because it is now a call to 0. Paul From pguyot@REDACTED Sun Jun 27 16:42:56 2010 From: pguyot@REDACTED (Paul Guyot) Date: Sun, 27 Jun 2010 16:42:56 +0200 Subject: Patch to fix hipe crash when calling a module with on_load Message-ID: <8ED0AD1A-6799-43CD-A485-F96F5636D683@kallisys.net> Hello, Here is a patch with a test case for the hipe crash bug when calling functions to a module with on_load : git fetch git://github.com/pguyot/otp.git pg/hipe_crash_with_on_load Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From fauli@REDACTED Mon Jun 28 14:38:44 2010 From: fauli@REDACTED (Christian Faulhammer) Date: Mon, 28 Jun 2010 14:38:44 +0200 Subject: Failures of Erlang 14A with LDFLAGS="-Wl,--as-needed" Message-ID: <20100628143844.18f0ea64@gentoo.org> Hi, when building with above linker flags on Linux, a build failure appears as shown in the attached build.log. Similar failures have been fixed in the past already. V-Li -- Christian Faulhammer, Gentoo Lisp project , #gentoo-lisp on FreeNode -------------- next part -------------- A non-text attachment was scrubbed... Name: erlang-build.log.bz2 Type: application/x-bzip Size: 10459 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From sverker@REDACTED Tue Jun 29 11:49:41 2010 From: sverker@REDACTED (Sverker Eriksson) Date: Tue, 29 Jun 2010 11:49:41 +0200 Subject: [erlang-bugs] [bug] erl_interface-3.6.5: C library erl_marshal, function erl_decode In-Reply-To: References: Message-ID: <4C29C1B5.4000602@erix.ericsson.se> Alexander Demidenko wrote: > Hi! > I found incorrect behavior of the function. > Easiest way to demonstrate in the following example: > dummy C code: > > * > * > > * byte * bb = new byte[10000]; > ETERM* t = erl_mk_int(-1250947744); > erl_encode(t, bb); > ETERM* d = erl_decode(bb); > DBGTERM("Result: ", d);* > > > Output: > *12:52:11 : Result: -1250947745* > > * > * > > > > *I suspect that the problem in compiler optimization options.* > > I found same bug in binary erl_interface by: > > Erlang R13B01 (erts-5.7.2) from standart ubuntu lucid repository. > > Erlang R13B04 (erts-5.7.5) from deb > http://ppa.launchpad.net/erlang-dev/ppa/ubuntu karmic main > > Thanks for reporting. The compiler was innocent, it did as it was told. Here's a quick patch: diff --git a/lib/erl_interface/src/legacy/erl_marshal.c b/lib/erl_interface/src/legacy/erl_marshal.c index c57c552..ab1953a 100644 --- a/lib/erl_interface/src/legacy/erl_marshal.c +++ b/lib/erl_interface/src/legacy/erl_marshal.c @@ -747,14 +747,11 @@ static ETERM *erl_decode_it(unsigned char **ext) return ep; } else if (arity == 4 && !((*ext)[3] & 0x80)) { /* It will fit into an int !! - * Note: It comes in "one's-complement notation" */ + i = (int) (((*ext)[3] << 24) | ((*ext)[2])<< 16 | + ((*ext)[1]) << 8 | (**ext)); if (sign) - i = (int) (~(((*ext)[3] << 24) | ((*ext)[2])<< 16 | - ((*ext)[1]) << 8 | (**ext)) | (unsigned int) sign); - else - i = (int) (((*ext)[3] << 24) | ((*ext)[2])<< 16 | - ((*ext)[1]) << 8 | (**ext)); + i = -i; ERL_TYPE(ep) = ERL_INTEGER; ep->uval.ival.i = i; *ext += arity; /Sverker, Erlang/OTP From alex.demidenko@REDACTED Tue Jun 29 14:27:40 2010 From: alex.demidenko@REDACTED (Alexander Demidenko) Date: Tue, 29 Jun 2010 19:27:40 +0700 Subject: [bug] syntax_tool-1.6.5, injury data in erl_syntax_lib:map() Message-ID: Hi, Try to show on a synthetic example: I have some source data: 1> X = {function,34,handle_event,2, [{clause,34, [{var,34,'_'},{var,34,'_'}],[], [{cons,35,{tuple,35,[]},{nil,36}}]}]}. {function,34,handle_event,2, ?? ? ? ? ?[{clause,34, ?? ? ? ? ? ? ? ? ? [{var,34,'_'},{var,34,'_'}], ?? ? ? ? ? ? ? ? ? [], ?? ? ? ? ? ? ? ? ? [{cons,35,{tuple,35,[]},{nil,36}}]}]} Now, I will try to traverse the data via erl_syntax_lib:map() 2> erl_syntax_lib:map(fun (El) -> io:format("~p~n", [erl_syntax:revert(El)]), El end, X). {atom,34,handle_event} {var,34,'_'} {var,34,'_'} {tuple,35,[]} {cons,35,{tuple,35,[]},{nil,35}} {clause,34,[{var,34,'_'},{var,34,'_'}],[],[{cons,35,{tuple,35,[]},{nil,35}}]} {function,34,handle_event,2, ?? ? ? ? ?[{clause,34, ?? ? ? ? ? ? ? ? ? [{var,34,'_'},{var,34,'_'}], ?? ? ? ? ? ? ? ? ? [], ?? ? ? ? ? ? ? ? ? [{cons,35,{tuple,35,[]},{nil,35}}]}]} {tree,function, ?? ? ?{attr,34,[],none}, ?? ? ?{function,{tree,atom,{attr,34,[],none},handle_event}, ?? ? ? ? ? ? ? ?[{tree,clause, ?? ? ? ? ? ? ? ? ? ? ? {attr,34,[],none}, ?? ? ? ? ? ? ? ? ? ? ? {clause,[{var,34,'_'},{var,34,'_'}], ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? none, ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [{tree,list, ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{attr,35,[],none}, ?? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{list,[{tuple,35,[]}],none}}]}}]}} Ops, erl_syntax_lib vodoo magic, make unexpected change original {nil,36} to {nil,35} To test, make control measurements via erl_syntax_lib:fold() 3> erl_syntax_lib:fold(fun (El, Acc) -> io:format("~p~n", [El]) end, [], X). {tree,atom,{attr,34,[],none},handle_event} {var,34,'_'} {var,34,'_'} {tuple,35,[]} {cons,35,{tuple,35,[]},{nil,36}} {clause,34,[{var,34,'_'},{var,34,'_'}],[],[{cons,35,{tuple,35,[]},{nil,36}}]} {function,34,handle_event,2, ?? ? ? ? ?[{clause,34, ?? ? ? ? ? ? ? ? ? [{var,34,'_'},{var,34,'_'}], ?? ? ? ? ? ? ? ? ? [], ?? ? ? ? ? ? ? ? ? [{cons,35,{tuple,35,[]},{nil,36}}]}]} ok Unlike map(), fold() working correctly. PS Test on Erlang R13B04 (erts-5.7.5) and Erlang R13B01 (erts-5.7.2) with same results -- --------------------------------------------- With best regards, Alexander. From mikpe@REDACTED Wed Jun 30 14:31:04 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 30 Jun 2010 14:31:04 +0200 Subject: fix native code crash when calling unloaded module with on_load function Message-ID: <19499.14600.813671.501500@pilspetsen.it.uu.se> As reported in erlang-bugs, the following sequence of events crashes the VM: 1. Module M1 is loaded and in native mode. 2. Module M2 is not loaded, in emulated mode, and has an on_load function. 3. M1 calls some function in M2. This works. 4. M1 again calls some function in M2. This segfaults. The reason for the crash is that when the beam loader fixes up export entries after a successful on_load function call, it erroneously clears the ->code[3] field in that module's export entries. This is redundant (no code in beam relies on ->code[3] being NULL), inconsistent with modules without on_load functions (there ->code[3] remains a valid beam instruction after the module is loaded), and breaks native code which needs the old ->address value in an export entry to remain valid after a module load step (before the load ->address points to ->code[3], after the load ->address points to the real code but uses of the old ->address value remain so ->code[3] must remain valid). Thus the fix for the crash is to simply not clear ->code[3]. This patch fixes R14A and should also fix R13B04. (There does exist a performance bug in this area, but it is unrelated to the on_load feature so will be fixed separately.) /Mikael Pettersson, HiPE --- otp_src_R14A/erts/emulator/beam/beam_bif_load.c.~1~ 2010-06-11 17:30:16.000000000 +0200 +++ otp_src_R14A/erts/emulator/beam/beam_bif_load.c 2010-06-30 13:17:03.000000000 +0200 @@ -337,7 +337,6 @@ BIF_RETTYPE finish_after_on_load_2(BIF_A ep->code[0] == BIF_ARG_1 && ep->code[4] != 0) { ep->address = (void *) ep->code[4]; - ep->code[3] = 0; ep->code[4] = 0; } } From pguyot@REDACTED Wed Jun 30 15:02:27 2010 From: pguyot@REDACTED (Paul Guyot) Date: Wed, 30 Jun 2010 15:02:27 +0200 Subject: fix native code crash when calling unloaded module with on_load function In-Reply-To: <19499.14600.813671.501500@pilspetsen.it.uu.se> References: <19499.14600.813671.501500@pilspetsen.it.uu.se> Message-ID: <804B40A0-0E09-441B-A03E-E4A30A62FAC9@kallisys.net> Le 30 juin 2010 ? 14:31, Mikael Pettersson a ?crit : > As reported in erlang-bugs, the following sequence of events crashes the VM: > > 1. Module M1 is loaded and in native mode. > 2. Module M2 is not loaded, in emulated mode, and has an on_load function. > 3. M1 calls some function in M2. This works. > 4. M1 again calls some function in M2. This segfaults. > > The reason for the crash is that when the beam loader fixes up export > entries after a successful on_load function call, it erroneously clears > the ->code[3] field in that module's export entries. This is redundant > (no code in beam relies on ->code[3] being NULL), inconsistent with > modules without on_load functions (there ->code[3] remains a valid beam > instruction after the module is loaded), and breaks native code which needs > the old ->address value in an export entry to remain valid after a module > load step (before the load ->address points to ->code[3], after the load > ->address points to the real code but uses of the old ->address value > remain so ->code[3] must remain valid). > > Thus the fix for the crash is to simply not clear ->code[3]. > This patch fixes R14A and should also fix R13B04. > > (There does exist a performance bug in this area, but it is unrelated > to the on_load feature so will be fixed separately.) Hello Mikael, Did you have a chance to check the patch I submitted to erlang-patch and which is available here: http://github.com/pguyot/otp/commit/495804b097aea4015e218d7b5da8d1372395580c My impression is that if we do not clear ep->code[3], this still points to call_error_handler. Instead of not clearing the value and relying to its initial assignation, I replaced the beam instruction to call_error_handler with a new beam instruction (call_from_hipe_stub) that simply jumps to the function. > --- otp_src_R14A/erts/emulator/beam/beam_bif_load.c.~1~ 2010-06-11 17:30:16.000000000 +0200 > +++ otp_src_R14A/erts/emulator/beam/beam_bif_load.c 2010-06-30 13:17:03.000000000 +0200 > @@ -337,7 +337,6 @@ BIF_RETTYPE finish_after_on_load_2(BIF_A > ep->code[0] == BIF_ARG_1 && > ep->code[4] != 0) { > ep->address = (void *) ep->code[4]; > - ep->code[3] = 0; > ep->code[4] = 0; > } > } Regards, Paul -- Semiocast http://semiocast.com/ +33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris From mikpe@REDACTED Wed Jun 30 15:24:22 2010 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 30 Jun 2010 15:24:22 +0200 Subject: fix native code crash when calling unloaded module with on_load function In-Reply-To: <804B40A0-0E09-441B-A03E-E4A30A62FAC9@kallisys.net> References: <19499.14600.813671.501500@pilspetsen.it.uu.se> <804B40A0-0E09-441B-A03E-E4A30A62FAC9@kallisys.net> Message-ID: <19499.17798.449872.208717@pilspetsen.it.uu.se> Paul Guyot writes: > Le 30 juin 2010 ? 14:31, Mikael Pettersson a ?crit : > > > As reported in erlang-bugs, the following sequence of events crashes the VM: > > > > 1. Module M1 is loaded and in native mode. > > 2. Module M2 is not loaded, in emulated mode, and has an on_load function. > > 3. M1 calls some function in M2. This works. > > 4. M1 again calls some function in M2. This segfaults. > > > > The reason for the crash is that when the beam loader fixes up export > > entries after a successful on_load function call, it erroneously clears > > the ->code[3] field in that module's export entries. This is redundant > > (no code in beam relies on ->code[3] being NULL), inconsistent with > > modules without on_load functions (there ->code[3] remains a valid beam > > instruction after the module is loaded), and breaks native code which needs > > the old ->address value in an export entry to remain valid after a module > > load step (before the load ->address points to ->code[3], after the load > > ->address points to the real code but uses of the old ->address value > > remain so ->code[3] must remain valid). > > > > Thus the fix for the crash is to simply not clear ->code[3]. > > This patch fixes R14A and should also fix R13B04. > > > > (There does exist a performance bug in this area, but it is unrelated > > to the on_load feature so will be fixed separately.) > > Hello Mikael, > > Did you have a chance to check the patch I submitted to erlang-patch and which is available here: > > http://github.com/pguyot/otp/commit/495804b097aea4015e218d7b5da8d1372395580c I did. It's way overkill for this specific bug. > My impression is that if we do not clear ep->code[3], this still points to call_error_handler. Instead of not clearing the value and relying to its initial assignation, I replaced the beam instruction to call_error_handler with a new beam instruction (call_from_hipe_stub) that simply jumps to the function. Your impression is correct, but it's not necessary to invent a new BEAM instruction to solve the on_load crash bug. Your approach is more related to my comment: > > (There does exist a performance bug in this area, but it is unrelated > > to the on_load feature so will be fixed separately.) But the performance bug exists whether or not on_load is used. And the solution I have in mind is very different from yours'. /Mikael From fauli@REDACTED Wed Jun 30 16:53:42 2010 From: fauli@REDACTED (Christian Faulhammer) Date: Wed, 30 Jun 2010 16:53:42 +0200 Subject: [erlang-bugs] Failures of Erlang 14A with LDFLAGS="-Wl,--as-needed" In-Reply-To: <20100628143844.18f0ea64@gentoo.org> References: <20100628143844.18f0ea64@gentoo.org> Message-ID: <20100630165342.02025712@gentoo.org> Hi, Christian Faulhammer : > when building with above linker flags on Linux, a build failure > appears as shown in the attached build.log. Similar failures have > been fixed in the past already. Attached is a patch to fix this issue, courtesy of Samuli Suominen of Gentoo project. V-Li -- Christian Faulhammer, Gentoo Lisp project , #gentoo-lisp on FreeNode -------------- next part -------------- A non-text attachment was scrubbed... Name: erlang-14.1-lm.patch Type: text/x-patch Size: 617 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From john.hughes@REDACTED Wed Jun 30 19:01:25 2010 From: john.hughes@REDACTED (John Hughes) Date: Wed, 30 Jun 2010 19:01:25 +0200 Subject: Funny behaviour from dets Message-ID: <594DB7712E254C5FA696BB49115A4E65@HALL> I'm playing with dets and QuickCheck, and have encountered behaviour that seems strange to me--although I'm not entirely clear on how dets is supposed to behave if the table is modified during a traversal, so I can't say for certain that this is a bug. Basically, I'm doing a lazy traversal of the table using match_object, and I'm finding that one of the objects in the table is being returned TWICE. To provoke that, I need to do two things: a.. insert an element with the same key (after starting the traversal) b.. do an apparently unrelated traversal of the table I've translated the QuickCheck counterexample into a unit test---which passes---so you can see the weirdness just by reading the code. Here's the test: test() -> % The file dets_table does not exist when we start the test dets:open_file(dets_table,[{type,bag}]), % Insert some data... note, 3 objects in the table, our match will % traverse 2 objects at a time. dets:insert(dets_table,[{0,0},{1,0},{2,3}]), % Fix the table before starting a match_object traversal. Note that % the behaviour is the same whether or not we do this--it's included % to show that the problem is not that I failed to fix the table! dets:safe_fixtable(dets_table,true), % Start the traversal, returning the first two objects. {[{1,0},{2,3}],Cont} = dets:match_object(dets_table,{'$1','$2'},2), % Add another object DURING THE TRAVERSAL. Important that it has % the SAME KEY as the object not yet traversed. dets:insert(dets_table,[{0,1}]), % Perform an apparently unrelated traversal of the table. [] = dets:match_object(dets_table,{2,2}), % Continue the original traversal. % WHAT??? How come we get {0,0} twice??? {[{0,0},{0,0},{0,1}],_} = dets:match_object(Cont), ok. The insertion and unrelated traversal can also be performed in another process--it makes no difference, the funny behaviour still occurs. Is this a bug? John