From sgolovan@REDACTED Sat Aug 1 15:35:28 2009 From: sgolovan@REDACTED (Sergei Golovan) Date: Sat, 1 Aug 2009 17:35:28 +0400 Subject: Patch for DNS over IPv6 support Message-ID: Hi! Appears that Erlang doesn't use DNS servers over IPv6. It ignores nameserver directive in /etc/resolv.conf if the IP it specifies is an IPv6 address and directly uses inet_udp and inet_tcp when connecting to DNS server (module inet_res). The problem is that if for some reason /etc/resolv.conf doesn't contain IPv4 nameservers the Erlang resolver doesn't work (see bug http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=539269 in Debian bugtracker). This situation isn't common now, but will be more and more common in the future. The attached patch adds transparent IPv6 support to inet_res module. It means that in case when nameserver IP address is an IPv4 address then it will be queried over IPv4, if it is an IPv6 address then it will be queried over IPv6. There is one drawback though: in case of DNS over UDP instead of opening a single socket and using it for queries to all nameservers the patched version opens one socket per DNS query. This may slightly decrease performance in case of many DNS servers in /etc/resolv.conf. Cheers! -- Sergei Golovan From sgolovan@REDACTED Sat Aug 1 16:52:46 2009 From: sgolovan@REDACTED (Sergei Golovan) Date: Sat, 1 Aug 2009 18:52:46 +0400 Subject: Patch for DNS over IPv6 support In-Reply-To: References: Message-ID: Seems like the mailing list gobbled the attachment. It is also attached to a message in Debian bugreport: http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=50;filename=resolver_ipv6.diff;att=1;bug=539269 On Sat, Aug 1, 2009 at 5:35 PM, Sergei Golovan wrote: > Hi! > > Appears that Erlang doesn't use DNS servers over IPv6. It ignores > nameserver directive in /etc/resolv.conf if the IP it specifies is an > IPv6 address and directly uses inet_udp and inet_tcp when connecting > to DNS server (module inet_res). > > The problem is that if for some reason /etc/resolv.conf doesn't > contain IPv4 nameservers the Erlang resolver doesn't work (see bug > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=539269 in Debian > bugtracker). This situation isn't common now, but will be more and > more common in the future. > > The attached patch adds transparent IPv6 support to inet_res module. > It means that in case when nameserver IP address is an IPv4 address > then it will be queried over IPv4, if it is an IPv6 address then it > will be queried over IPv6. There is one drawback though: in case of > DNS over UDP instead of opening a single socket and using it for > queries to all nameservers the patched version opens one socket per > DNS query. This may slightly decrease performance in case of many DNS > servers in /etc/resolv.conf. > > Cheers! > -- > Sergei Golovan > -- Sergei Golovan From akitada@REDACTED Sun Aug 9 07:03:47 2009 From: akitada@REDACTED (Akira Kitada) Date: Sun, 9 Aug 2009 14:03:47 +0900 Subject: Patch to make R13B01 build with non-C99 compiler Message-ID: <90bb445a0908082203n6db14c1ao631c28e9763902bf@mail.gmail.com> Hi, Unlike R12B-5, R13B01 doesn't compile with non-C99 compiler like gcc 2.95.4. The following patch replaces C99-ish code with good old C89. Note that I also needed Michael Turner's patch to build it with the old compiler. The patch can be found on http://www.erlang.org/cgi-bin/ezmlm-cgi?2:mss:1440:200907:dadanlkdcocjdfghmcaa diff --git a/lib/megaco/src/flex/megaco_flex_scanner_drv.flex.src b/lib/megaco/src/flex/megaco_flex_scanner_drv.flex.src index f8305db..b96a694 100644 --- a/lib/megaco/src/flex/megaco_flex_scanner_drv.flex.src +++ b/lib/megaco/src/flex/megaco_flex_scanner_drv.flex.src @@ -1021,8 +1021,6 @@ static void mfs_load_property_groups(MfsErlDrvData* dataP) // MfsErlDrvData* dataP = &mfs_drv_data; #endif - DBG( ("mfs_load_property_groups -> entry\n") ); - /* * Process the property groups string * v= is a property group delimiter @@ -1044,6 +1042,8 @@ static void mfs_load_property_groups(MfsErlDrvData* dataP) int valueStart = 0; // Start position of the property parm name int valueLen = 0; // Length of the value part + DBG( ("mfs_load_property_groups -> entry\n") ); + mfs_ensure_term_spec(dataP, 10); /*\xa0Just in case... */ while (i <= yyleng) { @@ -1498,14 +1498,14 @@ DRIVER_INIT(mfs_drv) static ErlDrvData mfs_start(ErlDrvPort port, char *buf) { - DBG( ("mfs_start -> entry\n") ); - #if defined(MEGACO_REENTRANT_FLEX_SCANNER) MfsErlDrvData* dataP = ALLOC(sizeof(MfsErlDrvData)); #else MfsErlDrvData* dataP = &mfs_drv_data; #endif + DBG( ("mfs_start -> entry\n") ); + dataP->port = port; dataP->digit_map_name_ptr = NULL; dataP->digit_map_name_len = 0; From kenji.rikitake@REDACTED Thu Aug 13 12:09:26 2009 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Thu, 13 Aug 2009 19:09:26 +0900 Subject: Erlang R13B01 ssh and crypto module patch of aes128-cbc support on ssh Message-ID: <20090813100926.GA12085@k2r.org> The following patch includes a feature enhancement of Erlang R13B01 ssh and crypto modules to support aes128-cbc mode on SSH. BUG FOUND: ssh_transport:unpack/3 causes crypto:aes_cbc_ivec/1 to crash, by passing a null binary (<<>>), which causes erlang:split_binary/2 to crash by badarg. By tracing the exchange between a FreeBSD OpenSSH implementation, I found a case where the internal variable SshLength in ssh_transport:unpack/3 goes to zero, which leads to passing a null binary as an argument to ssh_transport:decrypt_blocks/3 and to aes_cbc_ivec/1. So I added a case statement to avoid calling the decrypt_blocks/3 when SshLength = 0. Patches follow. Kenji Rikitake --- lib/ssh/src/ssh_transport.erl.FCS 2009-06-05 21:55:34.000000000 +0900 +++ lib/ssh/src/ssh_transport.erl 2009-08-13 17:42:28.000000000 +0900 @@ -228,8 +228,10 @@ cookie = Random, kex_algorithms = ["diffie-hellman-group1-sha1"], server_host_key_algorithms = ["ssh-rsa", "ssh-dss"], - encryption_algorithms_client_to_server = ["3des-cbc"], - encryption_algorithms_server_to_client = ["3des-cbc"], + %% encryption_algorithms_client_to_server = ["3des-cbc"], + %% encryption_algorithms_server_to_client = ["3des-cbc"], + encryption_algorithms_client_to_server = ["aes128-cbc","3des-cbc"], + encryption_algorithms_server_to_client = ["aes128-cbc","3des-cbc"], mac_algorithms_client_to_server = ["hmac-sha1"], mac_algorithms_server_to_client = ["hmac-sha1"], compression_algorithms_client_to_server = Compression, @@ -243,8 +245,10 @@ cookie = Random, kex_algorithms = ["diffie-hellman-group1-sha1"], server_host_key_algorithms = ["ssh-dss"], - encryption_algorithms_client_to_server = ["3des-cbc"], - encryption_algorithms_server_to_client = ["3des-cbc"], + %% encryption_algorithms_client_to_server = ["3des-cbc"], + %% encryption_algorithms_server_to_client = ["3des-cbc"], + encryption_algorithms_client_to_server = ["aes128-cbc","3des-cbc"], + encryption_algorithms_server_to_client = ["aes128-cbc","3des-cbc"], mac_algorithms_client_to_server = ["hmac-sha1"], mac_algorithms_server_to_client = ["hmac-sha1"], compression_algorithms_client_to_server = Compression, @@ -703,6 +707,9 @@ unpack(EncodedSoFar, ReminingLenght, #ssh{recv_mac_size = MacSize} = Ssh0) -> SshLength = ReminingLenght - MacSize, + %%?dbg(?DBG_CRYPTO, + %% "unpack: EncodedsoFar=~p SshLength=~p ReminingLenght=~p MacSize=~p ~n", + %% [EncodedSoFar, SshLength, ReminingLenght, MacSize]), {NoMac, Mac, Rest} = case MacSize of 0 -> <> = EncodedSoFar, {NoMac0, Mac0, Rest0} end, - {Ssh1, DecData, <<>>} = - ssh_transport:decrypt_blocks(NoMac, SshLength, Ssh0), + %%?dbg(?DBG_CRYPTO, "unpack: NoMac=~p Mac=~p Rest=~p ~n", + %% [NoMac, Mac, Rest]), + {Ssh1, DecData, <<>>} = case SshLength of + 0 -> + {Ssh0, <<>>, <<>>}; + _ -> + ssh_transport:decrypt_blocks(NoMac, SshLength, Ssh0) + end, + %%?dbg(?DBG_CRYPTO, "unpack: Ssh1=~p DecData=~p Rest=~p Mac=~p ~n", + %% [Ssh1, DecData, Rest, Mac]), {Ssh1, DecData, Rest, Mac}. msg_data(PacketData) -> @@ -800,6 +815,18 @@ <> = hash(Ssh, "D", 192), {ok, Ssh#ssh{encrypt_keys = {K1,K2,K3}, encrypt_block_size = 8, + encrypt_ctx = IV}}; +encrypt_init(#ssh{encrypt = 'aes128-cbc', role = client} = Ssh) -> + IV = hash(Ssh, "A", 128), + <> = hash(Ssh, "C", 128), + {ok, Ssh#ssh{encrypt_keys = K, + encrypt_block_size = 16, + encrypt_ctx = IV}}; +encrypt_init(#ssh{encrypt = 'aes128-cbc', role = server} = Ssh) -> + IV = hash(Ssh, "B", 128), + <> = hash(Ssh, "D", 128), + {ok, Ssh#ssh{encrypt_keys = K, + encrypt_block_size = 16, encrypt_ctx = IV}}. encrypt_final(Ssh) -> @@ -815,10 +842,19 @@ encrypt_keys = {K1,K2,K3}, encrypt_ctx = IV0} = Ssh, Data) -> %%?dbg(?DBG_CRYPTO, "encrypt: IV=~p K1=~p, K2=~p, K3=~p ~n", - %% [IV0,K1,K2,K3]), + %% [IV0,K1,K2,K3]), Enc = crypto:des3_cbc_encrypt(K1,K2,K3,IV0,Data), %%?dbg(?DBG_CRYPTO, "encrypt: ~p -> ~p ~n", [Data, Enc]), IV = crypto:des_cbc_ivec(Enc), + {Ssh#ssh{encrypt_ctx = IV}, Enc}; +encrypt(#ssh{encrypt = 'aes128-cbc', + encrypt_keys = K, + encrypt_ctx = IV0} = Ssh, Data) -> + %%?dbg(?DBG_CRYPTO, "encrypt: IV=~p K=~p ~n", + %% [IV0,K]), + Enc = crypto:aes_cbc_128_encrypt(K,IV0,Data), + %%?dbg(?DBG_CRYPTO, "encrypt: ~p -> ~p ~n", [Data, Enc]), + IV = crypto:aes_cbc_ivec(Enc), {Ssh#ssh{encrypt_ctx = IV}, Enc}. @@ -840,7 +876,21 @@ hash(Ssh, "C", 192)}, <> = KD, {ok, Ssh#ssh{decrypt_keys = {K1, K2, K3}, decrypt_ctx = IV, - decrypt_block_size = 8}}. + decrypt_block_size = 8}}; + +decrypt_init(#ssh{decrypt = 'aes128-cbc', role = client} = Ssh) -> + {IV, KD} = {hash(Ssh, "B", 128), + hash(Ssh, "D", 128)}, + <> = KD, + {ok, Ssh#ssh{decrypt_keys = K, decrypt_ctx = IV, + decrypt_block_size = 16}}; + +decrypt_init(#ssh{decrypt = 'aes128-cbc', role = server} = Ssh) -> + {IV, KD} = {hash(Ssh, "A", 128), + hash(Ssh, "C", 128)}, + <> = KD, + {ok, Ssh#ssh{decrypt_keys = K, decrypt_ctx = IV, + decrypt_block_size = 16}}. decrypt_final(Ssh) -> {ok, Ssh#ssh {decrypt = none, @@ -858,6 +908,14 @@ Dec = crypto:des3_cbc_decrypt(K1,K2,K3,IV0,Data), %%?dbg(?DBG_CRYPTO, "decrypt: ~p -> ~p ~n", [Data, Dec]), IV = crypto:des_cbc_ivec(Data), + {Ssh#ssh{decrypt_ctx = IV}, Dec}; +decrypt(#ssh{decrypt = 'aes128-cbc', decrypt_keys = Key, + decrypt_ctx = IV0} = Ssh, Data) -> + %%?dbg(?DBG_CRYPTO, "decrypt: IV=~p Key=~p ~n", + %% [IV0,Key]), + Dec = crypto:aes_cbc_128_decrypt(Key,IV0,Data), + %%?dbg(?DBG_CRYPTO, "decrypt: ~p -> ~p ~n", [Data, Dec]), + IV = crypto:aes_cbc_ivec(Data), {Ssh#ssh{decrypt_ctx = IV}, Dec}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% --- lib/crypto/src/crypto.erl.FCS 2009-03-12 21:28:35.000000000 +0900 +++ lib/crypto/src/crypto.erl 2009-08-13 13:27:09.000000000 +0900 @@ -45,6 +45,7 @@ %% -export([idea_cbc_encrypt/3, idea_cbc_decrypt/3]). -export([aes_cbc_128_encrypt/3, aes_cbc_128_decrypt/3]). -export([aes_cbc_256_encrypt/3, aes_cbc_256_decrypt/3]). +-export([aes_cbc_ivec/1]). -export([dh_generate_parameters/2, dh_check/1]). %% Testing see below @@ -469,6 +470,19 @@ control(?AES_CBC_256_DECRYPT, [Key, IVec, Data]). %% +%% aes_cbc_ivec(Data) -> binary() +%% +%% Returns the IVec to be used in the next iteration of +%% aes_cbc_*_[encrypt|decrypt]. +%% IVec size: 16 bytes +%% +aes_cbc_ivec(Data) when is_binary(Data) -> + {_, IVec} = split_binary(Data, size(Data) - 16), + IVec; +aes_cbc_ivec(Data) when is_list(Data) -> + aes_cbc_ivec(list_to_binary(Data)). + +%% %% XOR - xor to iolists and return a binary %% NB doesn't check that they are the same size, just concatenates %% them and sends them to the driver -------------- next part -------------- --- lib/ssh/src/ssh_transport.erl.FCS 2009-06-05 21:55:34.000000000 +0900 +++ lib/ssh/src/ssh_transport.erl 2009-08-13 17:42:28.000000000 +0900 @@ -228,8 +228,10 @@ cookie = Random, kex_algorithms = ["diffie-hellman-group1-sha1"], server_host_key_algorithms = ["ssh-rsa", "ssh-dss"], - encryption_algorithms_client_to_server = ["3des-cbc"], - encryption_algorithms_server_to_client = ["3des-cbc"], + %% encryption_algorithms_client_to_server = ["3des-cbc"], + %% encryption_algorithms_server_to_client = ["3des-cbc"], + encryption_algorithms_client_to_server = ["aes128-cbc","3des-cbc"], + encryption_algorithms_server_to_client = ["aes128-cbc","3des-cbc"], mac_algorithms_client_to_server = ["hmac-sha1"], mac_algorithms_server_to_client = ["hmac-sha1"], compression_algorithms_client_to_server = Compression, @@ -243,8 +245,10 @@ cookie = Random, kex_algorithms = ["diffie-hellman-group1-sha1"], server_host_key_algorithms = ["ssh-dss"], - encryption_algorithms_client_to_server = ["3des-cbc"], - encryption_algorithms_server_to_client = ["3des-cbc"], + %% encryption_algorithms_client_to_server = ["3des-cbc"], + %% encryption_algorithms_server_to_client = ["3des-cbc"], + encryption_algorithms_client_to_server = ["aes128-cbc","3des-cbc"], + encryption_algorithms_server_to_client = ["aes128-cbc","3des-cbc"], mac_algorithms_client_to_server = ["hmac-sha1"], mac_algorithms_server_to_client = ["hmac-sha1"], compression_algorithms_client_to_server = Compression, @@ -703,6 +707,9 @@ unpack(EncodedSoFar, ReminingLenght, #ssh{recv_mac_size = MacSize} = Ssh0) -> SshLength = ReminingLenght - MacSize, + %%?dbg(?DBG_CRYPTO, + %% "unpack: EncodedsoFar=~p SshLength=~p ReminingLenght=~p MacSize=~p ~n", + %% [EncodedSoFar, SshLength, ReminingLenght, MacSize]), {NoMac, Mac, Rest} = case MacSize of 0 -> <> = EncodedSoFar, {NoMac0, Mac0, Rest0} end, - {Ssh1, DecData, <<>>} = - ssh_transport:decrypt_blocks(NoMac, SshLength, Ssh0), + %%?dbg(?DBG_CRYPTO, "unpack: NoMac=~p Mac=~p Rest=~p ~n", + %% [NoMac, Mac, Rest]), + {Ssh1, DecData, <<>>} = case SshLength of + 0 -> + {Ssh0, <<>>, <<>>}; + _ -> + ssh_transport:decrypt_blocks(NoMac, SshLength, Ssh0) + end, + %%?dbg(?DBG_CRYPTO, "unpack: Ssh1=~p DecData=~p Rest=~p Mac=~p ~n", + %% [Ssh1, DecData, Rest, Mac]), {Ssh1, DecData, Rest, Mac}. msg_data(PacketData) -> @@ -800,6 +815,18 @@ <> = hash(Ssh, "D", 192), {ok, Ssh#ssh{encrypt_keys = {K1,K2,K3}, encrypt_block_size = 8, + encrypt_ctx = IV}}; +encrypt_init(#ssh{encrypt = 'aes128-cbc', role = client} = Ssh) -> + IV = hash(Ssh, "A", 128), + <> = hash(Ssh, "C", 128), + {ok, Ssh#ssh{encrypt_keys = K, + encrypt_block_size = 16, + encrypt_ctx = IV}}; +encrypt_init(#ssh{encrypt = 'aes128-cbc', role = server} = Ssh) -> + IV = hash(Ssh, "B", 128), + <> = hash(Ssh, "D", 128), + {ok, Ssh#ssh{encrypt_keys = K, + encrypt_block_size = 16, encrypt_ctx = IV}}. encrypt_final(Ssh) -> @@ -815,10 +842,19 @@ encrypt_keys = {K1,K2,K3}, encrypt_ctx = IV0} = Ssh, Data) -> %%?dbg(?DBG_CRYPTO, "encrypt: IV=~p K1=~p, K2=~p, K3=~p ~n", - %% [IV0,K1,K2,K3]), + %% [IV0,K1,K2,K3]), Enc = crypto:des3_cbc_encrypt(K1,K2,K3,IV0,Data), %%?dbg(?DBG_CRYPTO, "encrypt: ~p -> ~p ~n", [Data, Enc]), IV = crypto:des_cbc_ivec(Enc), + {Ssh#ssh{encrypt_ctx = IV}, Enc}; +encrypt(#ssh{encrypt = 'aes128-cbc', + encrypt_keys = K, + encrypt_ctx = IV0} = Ssh, Data) -> + %%?dbg(?DBG_CRYPTO, "encrypt: IV=~p K=~p ~n", + %% [IV0,K]), + Enc = crypto:aes_cbc_128_encrypt(K,IV0,Data), + %%?dbg(?DBG_CRYPTO, "encrypt: ~p -> ~p ~n", [Data, Enc]), + IV = crypto:aes_cbc_ivec(Enc), {Ssh#ssh{encrypt_ctx = IV}, Enc}. @@ -840,7 +876,21 @@ hash(Ssh, "C", 192)}, <> = KD, {ok, Ssh#ssh{decrypt_keys = {K1, K2, K3}, decrypt_ctx = IV, - decrypt_block_size = 8}}. + decrypt_block_size = 8}}; + +decrypt_init(#ssh{decrypt = 'aes128-cbc', role = client} = Ssh) -> + {IV, KD} = {hash(Ssh, "B", 128), + hash(Ssh, "D", 128)}, + <> = KD, + {ok, Ssh#ssh{decrypt_keys = K, decrypt_ctx = IV, + decrypt_block_size = 16}}; + +decrypt_init(#ssh{decrypt = 'aes128-cbc', role = server} = Ssh) -> + {IV, KD} = {hash(Ssh, "A", 128), + hash(Ssh, "C", 128)}, + <> = KD, + {ok, Ssh#ssh{decrypt_keys = K, decrypt_ctx = IV, + decrypt_block_size = 16}}. decrypt_final(Ssh) -> {ok, Ssh#ssh {decrypt = none, @@ -858,6 +908,14 @@ Dec = crypto:des3_cbc_decrypt(K1,K2,K3,IV0,Data), %%?dbg(?DBG_CRYPTO, "decrypt: ~p -> ~p ~n", [Data, Dec]), IV = crypto:des_cbc_ivec(Data), + {Ssh#ssh{decrypt_ctx = IV}, Dec}; +decrypt(#ssh{decrypt = 'aes128-cbc', decrypt_keys = Key, + decrypt_ctx = IV0} = Ssh, Data) -> + %%?dbg(?DBG_CRYPTO, "decrypt: IV=~p Key=~p ~n", + %% [IV0,Key]), + Dec = crypto:aes_cbc_128_decrypt(Key,IV0,Data), + %%?dbg(?DBG_CRYPTO, "decrypt: ~p -> ~p ~n", [Data, Dec]), + IV = crypto:aes_cbc_ivec(Data), {Ssh#ssh{decrypt_ctx = IV}, Dec}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -------------- next part -------------- --- lib/crypto/src/crypto.erl.FCS 2009-03-12 21:28:35.000000000 +0900 +++ lib/crypto/src/crypto.erl 2009-08-13 13:27:09.000000000 +0900 @@ -45,6 +45,7 @@ %% -export([idea_cbc_encrypt/3, idea_cbc_decrypt/3]). -export([aes_cbc_128_encrypt/3, aes_cbc_128_decrypt/3]). -export([aes_cbc_256_encrypt/3, aes_cbc_256_decrypt/3]). +-export([aes_cbc_ivec/1]). -export([dh_generate_parameters/2, dh_check/1]). %% Testing see below @@ -469,6 +470,19 @@ control(?AES_CBC_256_DECRYPT, [Key, IVec, Data]). %% +%% aes_cbc_ivec(Data) -> binary() +%% +%% Returns the IVec to be used in the next iteration of +%% aes_cbc_*_[encrypt|decrypt]. +%% IVec size: 16 bytes +%% +aes_cbc_ivec(Data) when is_binary(Data) -> + {_, IVec} = split_binary(Data, size(Data) - 16), + IVec; +aes_cbc_ivec(Data) when is_list(Data) -> + aes_cbc_ivec(list_to_binary(Data)). + +%% %% XOR - xor to iolists and return a binary %% NB doesn't check that they are the same size, just concatenates %% them and sends them to the driver From pfisher@REDACTED Fri Aug 14 16:54:27 2009 From: pfisher@REDACTED (Paul Fisher) Date: Fri, 14 Aug 2009 09:54:27 -0500 Subject: [PATCH] R13(B02) 64-bit opensolaris support Message-ID: <4A857AA3.4070107@alertlogic.net> I have finally had a chance to test 64-bit R13 in our environment and on opensolaris. The --enable-m64-build option added in R13 has helped the size of the changes, but unfortunately I still need the patch at the end of this mail to build a fully working 64-bit environment. When configuring/build like so: $ CC=gcc LD=gcc CFLAGS=-O2 LIBS=-lmtmalloc \ ./configure --enable-threads --enable-smp-support \ --enable-kernel-poll --enable-hipe \ --disable-megaco-flex-scanner-drvlineno --without-unixodbc \ --enable-m64-build The following issues exist: 1) --enable-m64-build disables hipe unconditionally To my knowledge HiPE works on amd64 solaris 5.11, so disabling this no matter what seems wrong. 2) DED_LDFLAGS is set for old solaris ld and not gnu ld or gcc The erts/configure(.in) should detect if gcc is being specified as the linker and set the DED_LDFLAGS appropriately, even though DED_LD gets set to $(CC) unconditionally. Things need to be sensitive to the options differences. 3) erl_interface configure.in does not understand --enable-m64-build The option needs to be added and the CFLAGS twiddled in the same way as erts/configure.in. Here is the patch against R13B02 Aug 12 snapshot that I ended up with (I included the changes to the generated files too, just for convenience): --- otp_src_R13B02/erts/configure 2009-08-11 13:01:22.000000000 -0500 +++ otp_src_R13B02/erts/configure.new 2009-08-13 15:54:04.243098019 -0500 @@ -2807,8 +2807,10 @@ { (exit 1); exit 1; }; } fi fi -if test X${enable_darwin_64bit} = Xyes -o X${enable_m64_build} = Xyes; then +if test X${enable_darwin_64bit} = Xyes; then enable_hipe=no +fi +if test X${enable_darwin_64bit} = Xyes -o X${enable_m64_build} = Xyes; then case $CFLAGS in *-m64*) ;; @@ -20897,9 +20899,16 @@ DED_LD_FLAG_RUNTIME_LIBRARY_PATH= ;; solaris2*|sysv4*) - DED_LDFLAGS="-G" - if test X${enable_m64_build} = Xyes; then - DED_LDFLAGS="-64 $DED_LDFLAGS" + if test "x$GCC" = "xyes"; then + DED_LD=gcc + DED_LDFLAGS="-shared" + DED_LDFLAGS_64="-m64" + else + DED_LDFLAGS="-G" + DED_LDFLAGS_64="-64" + fi + if test "X${enable_m64_build}" = "Xyes"; then + DED_LDFLAGS="$DED_LDFLAGS_64 $DED_LDFLAGS" fi ;; aix4*) --- otp_src_R13B02/erts/configure.in 2009-06-05 07:53:07.000000000 -0500 +++ otp_src_R13B02/erts/configure.in.new 2009-08-13 15:54:04.242383252 -0500 @@ -291,8 +291,10 @@ AC_MSG_ERROR([--enable-darwin-64bit only supported on x86 host]) fi fi -if test X${enable_darwin_64bit} = Xyes -o X${enable_m64_build} = Xyes; then +if test X${enable_darwin_64bit} = Xyes; then enable_hipe=no +fi +if test X${enable_darwin_64bit} = Xyes -o X${enable_m64_build} = Xyes; then case $CFLAGS in *-m64*) ;; @@ -2924,9 +2926,17 @@ DED_LD_FLAG_RUNTIME_LIBRARY_PATH= ;; solaris2*|sysv4*) - DED_LDFLAGS="-G" - if test X${enable_m64_build} = Xyes; then - DED_LDFLAGS="-64 $DED_LDFLAGS" + if test "x$GCC" = "xyes"; then + DED_LD=gcc + DED_LDFLAGS="-shared" + DED_LDFLAGS_64="-m64" + + elif + DED_LDFLAGS="-G" + DED_LDFLAGS_64="-m64" + fi + if test "X${enable_m64_build}" = "Xyes"; then + DED_LDFLAGS="$DED_LDFLAGS_64 $DED_LDFLAGS" fi ;; aix4*) --- otp_src_R13B02/lib/erl_interface/configure 2009-08-11 13:01:14.000000000 -0500 +++ otp_src_R13B02/lib/erl_interface/configure.new 2009-08-14 08:20:24.804342800 -0500 @@ -1440,6 +1440,27 @@ threads_disabled=no fi; +# Check whether --enable-m64-build or --disable-m64-build was given. +if test "${enable_m64_build+set}" = set; then + enableval="$enable_m64_build" + case "$enableval" in + no) enable_m64_build=no ;; + *) enable_m64_build=yes ;; + esac +else + enable_m64_build=no +fi; + +if test X${enable_m64_build} = Xyes; then + case $CFLAGS in + *-m64*) + ;; + *) + CFLAGS="-m64 $CFLAGS" + ;; + esac +fi + ac_ext=c ac_cpp='$CPP $CPPFLAGS' --- otp_src_R13B02/lib/erl_interface/configure.in 2009-08-14 08:24:02.056578225 -0500 +++ otp_src_R13B02/lib/erl_interface/configure.in.new 2009-08-14 08:25:16.412178147 -0500 @@ -87,6 +87,24 @@ esac ], [ threads_disabled=no ]) +AC_ARG_ENABLE(m64-build, +[ --enable-m64-build build 64bit binaries using the -m64 flag to (g)cc], +[ case "$enableval" in + no) enable_m64_build=no ;; + *) enable_m64_build=yes ;; + esac +],enable_m64_build=no) + +if test X${enable_m64_build} = Xyes; then + case $CFLAGS in + *-m64*) + ;; + *) + CFLAGS="-m64 $CFLAGS" + ;; + esac +fi + dnl ---------------------------------------------------------------------- dnl Checks for programs dnl ---------------------------------------------------------------------- From yrashk@REDACTED Mon Aug 17 15:36:22 2009 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Mon, 17 Aug 2009 06:36:22 -0700 Subject: PATCH: R13B01: new() function with module parameter variable(s) in guard crashes compiler Message-ID: <674D0626-718F-4ACB-B50A-64506E9C6E17@gmail.com> Hello, Currently, R13B01's compiler crashes on source code like this: -module(somemodule,[P1]). -compile(export_all). new() when P1 > 0 -> instance(hello). The crash looks like this: ./somemodule.erl:none: internal error in core_dsetel_module; crash reason: {{badmatch,error}, [{sys_core_dsetel,visit,2}, {lists,mapfoldl,3}, {sys_core_dsetel,visit,2}, {sys_core_dsetel,visit,2}, {lists,mapfoldl,3}, {sys_core_dsetel,visit,2}, {sys_core_dsetel,visit,2}, {sys_core_dsetel,visit_module_1,3}]} Attached you can find a quick patch that makes compiler emit an error instead: ./somemodule.erl:4: variable 'P1' is unbound This patch is also available through this branch: http://github.com/yrashk/erlang/commits/compiler-new-guards-use-of-parameters The solution in this patch might not be ideal, but it is a start; if OTP team will be interested in including it into official source code tree, I will be happy to spend a little bit more time to make it somewhat nicer. Thanks, Yurii. -------------- next part -------------- A non-text attachment was scrubbed... Name: compiler-new-guards-use-of-parameters.diff Type: application/octet-stream Size: 930 bytes Desc: not available URL: -------------- next part -------------- From yrashk@REDACTED Mon Aug 17 15:51:09 2009 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Mon, 17 Aug 2009 06:51:09 -0700 Subject: [erlang-patches] PATCH: R13B01: new() function with module parameter variable(s) in guard crashes compiler In-Reply-To: <674D0626-718F-4ACB-B50A-64506E9C6E17@gmail.com> References: <674D0626-718F-4ACB-B50A-64506E9C6E17@gmail.com> Message-ID: Hello again, In my previous message I included a patch that goes in a reverse direction. Please find proper version attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: compiler-new-guards-use-of-parameters.diff Type: application/octet-stream Size: 930 bytes Desc: not available URL: -------------- next part -------------- Yurii. On 17-Aug-09, at 6:36 AM, Yurii Rashkovskii wrote: > Hello, > > Currently, R13B01's compiler crashes on source code like this: > > -module(somemodule,[P1]). > -compile(export_all). > > new() when P1 > 0 -> > instance(hello). > > The crash looks like this: > > ./somemodule.erl:none: internal error in core_dsetel_module; > crash reason: {{badmatch,error}, > [{sys_core_dsetel,visit,2}, > {lists,mapfoldl,3}, > {sys_core_dsetel,visit,2}, > {sys_core_dsetel,visit,2}, > {lists,mapfoldl,3}, > {sys_core_dsetel,visit,2}, > {sys_core_dsetel,visit,2}, > {sys_core_dsetel,visit_module_1,3}]} > > Attached you can find a quick patch that makes compiler emit an > error instead: > > ./somemodule.erl:4: variable 'P1' is unbound > > This patch is also available through this branch: http://github.com/yrashk/erlang/commits/compiler-new-guards-use-of-parameters > > The solution in this patch might not be ideal, but it is a start; if > OTP team will be interested in including it into official source > code tree, I will be happy to spend a little bit more time to make > it somewhat nicer. > > Thanks, > Yurii. > > > > > ________________________________________________________________ > erlang-patches mailing list. See http://www.erlang.org/faq.html > erlang-patches (at) erlang.org From egil@REDACTED Mon Aug 17 12:01:00 2009 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Mon, 17 Aug 2009 12:01:00 +0200 Subject: [erlang-patches] [PATCH] R13(B02) 64-bit opensolaris support In-Reply-To: <4A857AA3.4070107@alertlogic.net> References: <4A857AA3.4070107@alertlogic.net> Message-ID: <4A892A5C.80609@erix.ericsson.se> Hi Paul, Thank you for reporting this issue, I will have a look at it. Regards, Bj?rn-Egil Erlang/OTP Paul Fisher wrote: > I have finally had a chance to test 64-bit R13 in our environment and on > opensolaris. The --enable-m64-build option added in R13 has helped the > size of the changes, but unfortunately I still need the patch at the end > of this mail to build a fully working 64-bit environment. > > When configuring/build like so: > > $ CC=gcc LD=gcc CFLAGS=-O2 LIBS=-lmtmalloc \ > ./configure --enable-threads --enable-smp-support \ > --enable-kernel-poll --enable-hipe \ > --disable-megaco-flex-scanner-drvlineno --without-unixodbc \ > --enable-m64-build > > The following issues exist: > > 1) --enable-m64-build disables hipe unconditionally > To my knowledge HiPE works on amd64 solaris 5.11, so disabling this no > matter what seems wrong. > > 2) DED_LDFLAGS is set for old solaris ld and not gnu ld or gcc > The erts/configure(.in) should detect if gcc is being specified as the > linker and set the DED_LDFLAGS appropriately, even though DED_LD gets > set to $(CC) unconditionally. Things need to be sensitive to the > options differences. > > 3) erl_interface configure.in does not understand --enable-m64-build > The option needs to be added and the CFLAGS twiddled in the same way as > erts/configure.in. > > Here is the patch against R13B02 Aug 12 snapshot that I ended up with (I > included the changes to the generated files too, just for convenience): > > --- otp_src_R13B02/erts/configure 2009-08-11 13:01:22.000000000 -0500 > +++ otp_src_R13B02/erts/configure.new 2009-08-13 15:54:04.243098019 > -0500 > @@ -2807,8 +2807,10 @@ > { (exit 1); exit 1; }; } > fi > fi > -if test X${enable_darwin_64bit} = Xyes -o X${enable_m64_build} = Xyes; > then > +if test X${enable_darwin_64bit} = Xyes; then > enable_hipe=no > +fi > +if test X${enable_darwin_64bit} = Xyes -o X${enable_m64_build} = Xyes; > then > case $CFLAGS in > *-m64*) > ;; > @@ -20897,9 +20899,16 @@ > DED_LD_FLAG_RUNTIME_LIBRARY_PATH= > ;; > solaris2*|sysv4*) > - DED_LDFLAGS="-G" > - if test X${enable_m64_build} = Xyes; then > - DED_LDFLAGS="-64 $DED_LDFLAGS" > + if test "x$GCC" = "xyes"; then > + DED_LD=gcc > + DED_LDFLAGS="-shared" > + DED_LDFLAGS_64="-m64" > + else > + DED_LDFLAGS="-G" > + DED_LDFLAGS_64="-64" > + fi > + if test "X${enable_m64_build}" = "Xyes"; then > + DED_LDFLAGS="$DED_LDFLAGS_64 $DED_LDFLAGS" > fi > ;; > aix4*) > --- otp_src_R13B02/erts/configure.in 2009-06-05 07:53:07.000000000 -0500 > +++ otp_src_R13B02/erts/configure.in.new 2009-08-13 > 15:54:04.242383252 -0500 > @@ -291,8 +291,10 @@ > AC_MSG_ERROR([--enable-darwin-64bit only supported on x86 host]) > fi > fi > -if test X${enable_darwin_64bit} = Xyes -o X${enable_m64_build} = Xyes; > then > +if test X${enable_darwin_64bit} = Xyes; then > enable_hipe=no > +fi > +if test X${enable_darwin_64bit} = Xyes -o X${enable_m64_build} = Xyes; > then > case $CFLAGS in > *-m64*) > ;; > @@ -2924,9 +2926,17 @@ > DED_LD_FLAG_RUNTIME_LIBRARY_PATH= > ;; > solaris2*|sysv4*) > - DED_LDFLAGS="-G" > - if test X${enable_m64_build} = Xyes; then > - DED_LDFLAGS="-64 $DED_LDFLAGS" > + if test "x$GCC" = "xyes"; then > + DED_LD=gcc > + DED_LDFLAGS="-shared" > + DED_LDFLAGS_64="-m64" > + > + elif > + DED_LDFLAGS="-G" > + DED_LDFLAGS_64="-m64" > + fi > + if test "X${enable_m64_build}" = "Xyes"; then > + DED_LDFLAGS="$DED_LDFLAGS_64 $DED_LDFLAGS" > fi > ;; > aix4*) > --- otp_src_R13B02/lib/erl_interface/configure 2009-08-11 > 13:01:14.000000000 -0500 > +++ otp_src_R13B02/lib/erl_interface/configure.new 2009-08-14 > 08:20:24.804342800 -0500 > @@ -1440,6 +1440,27 @@ > threads_disabled=no > fi; > > +# Check whether --enable-m64-build or --disable-m64-build was given. > +if test "${enable_m64_build+set}" = set; then > + enableval="$enable_m64_build" > + case "$enableval" in > + no) enable_m64_build=no ;; > + *) enable_m64_build=yes ;; > + esac > +else > + enable_m64_build=no > +fi; > + > +if test X${enable_m64_build} = Xyes; then > + case $CFLAGS in > + *-m64*) > + ;; > + *) > + CFLAGS="-m64 $CFLAGS" > + ;; > + esac > +fi > + > > ac_ext=c > ac_cpp='$CPP $CPPFLAGS' > --- otp_src_R13B02/lib/erl_interface/configure.in 2009-08-14 > 08:24:02.056578225 -0500 > +++ otp_src_R13B02/lib/erl_interface/configure.in.new 2009-08-14 > 08:25:16.412178147 -0500 > @@ -87,6 +87,24 @@ > esac ], > [ threads_disabled=no ]) > > +AC_ARG_ENABLE(m64-build, > +[ --enable-m64-build build 64bit binaries using the -m64 flag to > (g)cc], > +[ case "$enableval" in > + no) enable_m64_build=no ;; > + *) enable_m64_build=yes ;; > + esac > +],enable_m64_build=no) > + > +if test X${enable_m64_build} = Xyes; then > + case $CFLAGS in > + *-m64*) > + ;; > + *) > + CFLAGS="-m64 $CFLAGS" > + ;; > + esac > +fi > + > dnl ---------------------------------------------------------------------- > dnl Checks for programs > dnl ---------------------------------------------------------------------- > > > ________________________________________________________________ > erlang-patches mailing list. See http://www.erlang.org/faq.html > erlang-patches (at) erlang.org From yrashk@REDACTED Mon Aug 17 18:37:18 2009 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Mon, 17 Aug 2009 09:37:18 -0700 Subject: [erlang-patches] PATCH: R13B01: new() function with module parameter variable(s) in guard crashes compiler In-Reply-To: References: <674D0626-718F-4ACB-B50A-64506E9C6E17@gmail.com> Message-ID: <67D6FBF2-13D4-4F85-B76B-61E8308BE079@gmail.com> Hello again :) I am very sorry, but I found a bug in my patch. You can find new patch attached. -------------- next part -------------- A non-text attachment was scrubbed... Name: compiler-new-guards-use-of-parameters.diff Type: application/octet-stream Size: 962 bytes Desc: not available URL: -------------- next part -------------- On 17-Aug-09, at 6:51 AM, Yurii Rashkovskii wrote: > Hello again, > > In my previous message I included a patch that goes in a reverse > direction. Please find proper version attached. > > > > > Yurii. > > On 17-Aug-09, at 6:36 AM, Yurii Rashkovskii wrote: > >> Hello, >> >> Currently, R13B01's compiler crashes on source code like this: >> >> -module(somemodule,[P1]). >> -compile(export_all). >> >> new() when P1 > 0 -> >> instance(hello). >> >> The crash looks like this: >> >> ./somemodule.erl:none: internal error in core_dsetel_module; >> crash reason: {{badmatch,error}, >> [{sys_core_dsetel,visit,2}, >> {lists,mapfoldl,3}, >> {sys_core_dsetel,visit,2}, >> {sys_core_dsetel,visit,2}, >> {lists,mapfoldl,3}, >> {sys_core_dsetel,visit,2}, >> {sys_core_dsetel,visit,2}, >> {sys_core_dsetel,visit_module_1,3}]} >> >> Attached you can find a quick patch that makes compiler emit an >> error instead: >> >> ./somemodule.erl:4: variable 'P1' is unbound >> >> This patch is also available through this branch: http://github.com/yrashk/erlang/commits/compiler-new-guards-use-of-parameters >> >> The solution in this patch might not be ideal, but it is a start; >> if OTP team will be interested in including it into official source >> code tree, I will be happy to spend a little bit more time to make >> it somewhat nicer. >> >> Thanks, >> Yurii. >> >> >> >> >> ________________________________________________________________ >> erlang-patches mailing list. See http://www.erlang.org/faq.html >> erlang-patches (at) erlang.org > > > > ________________________________________________________________ > erlang-patches mailing list. See http://www.erlang.org/faq.html > erlang-patches (at) erlang.org From davidepesa@REDACTED Mon Aug 17 23:03:21 2009 From: davidepesa@REDACTED (Davide Pesavento) Date: Mon, 17 Aug 2009 23:03:21 +0200 Subject: Build system ignores LDFLAGS In-Reply-To: <2da21fe50908161305n6b3ce3cam3283ed47fadf91e6@mail.gmail.com> References: <2da21fe50908161305n6b3ce3cam3283ed47fadf91e6@mail.gmail.com> Message-ID: <2da21fe50908171403n58809a00ob0eb59d5866a3007@mail.gmail.com> On Mon, Apr 20, 2009 at 14:30, Raimo Niskanen wrote: > On Fri, Apr 17, 2009 at 11:34:49PM +0200, Davide Pesavento wrote: >> On Sat, Apr 4, 2009 at 13:05, Davide Pesavento wrote: >> > On Tue, Mar 31, 2009 at 22:13, Davide Pesavento wrote: >> >> Hello, >> >> >> >> Erlang's build system seems to ignore custom LDFLAGS. This is the QA >> >> warning shown by Portage (Gentoo Linux's package manager) after >> >> building R12B-5 with LDFLAGS="-Wl,-O1 -Wl,--as-needed >> >> -Wl,--hash-style=gnu" >> >> >> >> QA Notice: Files built without respecting LDFLAGS have been detected >> >> ?Please include the following list of files in your report: >> >> /usr/lib64/erlang/lib/runtime_tools-1.7.3/priv/lib/trace_file_drv.so >> >> /usr/lib64/erlang/lib/runtime_tools-1.7.3/priv/lib/trace_ip_drv.so >> >> /usr/lib64/erlang/lib/megaco-3.9.1.1/priv/lib/megaco_flex_scanner_drv_mt.so >> >> /usr/lib64/erlang/lib/megaco-3.9.1.1/priv/lib/megaco_flex_scanner_drv.so >> >> /usr/lib64/erlang/lib/erl_interface-3.5.9/bin/erl_call >> >> /usr/lib64/erlang/lib/ssl-3.10/priv/bin/ssl_esock >> >> /usr/lib64/erlang/lib/asn1-1.6.2/priv/lib/asn1_erl_drv.so >> >> /usr/lib64/erlang/lib/crypto-1.5.3/priv/lib/crypto_drv.so >> >> /usr/lib64/erlang/lib/common_test-1.3.4/priv/lib/erl_rx_driver.so >> >> /usr/lib64/erlang/erts-5.6.5/bin/child_setup >> >> >> >> Before digging into the Makefiles, I'd like to know if you already >> >> have some ideas about the root cause of this issue. >> >> >> > >> > This bug is still present in R13A. >> > >> > ?* QA Notice: Files built without respecting LDFLAGS have been detected >> > ?* ?Please include the following list of files in your report: >> > ?* /usr/lib64/erlang/lib/runtime_tools-1.8/priv/lib/trace_file_drv.so >> > ?* /usr/lib64/erlang/lib/runtime_tools-1.8/priv/lib/trace_ip_drv.so >> > ?* /usr/lib64/erlang/lib/erl_interface-3.6/bin/erl_call >> > ?* /usr/lib64/erlang/lib/crypto-1.6/priv/lib/crypto_drv.so >> > ?* /usr/lib64/erlang/lib/megaco-3.10.0.1/priv/lib/megaco_flex_scanner_drv_mt.so >> > ?* /usr/lib64/erlang/lib/megaco-3.10.0.1/priv/lib/megaco_flex_scanner_drv.so >> > ?* /usr/lib64/erlang/lib/ssl-3.10.1/priv/bin/ssl_esock >> > ?* /usr/lib64/erlang/lib/asn1-1.6.8/priv/lib/asn1_erl_drv.so >> > ?* /usr/lib64/erlang/erts-5.7/bin/child_setup >> > >> > Best regards, >> > Davide >> > >> >> The attached patch fixes the problem for me: LDFLAGS are now respected. >> >> Regards, >> Davide > > The patch looks great! It is unfortunately a few days late to make > it into R13B. Now the source tree is labeled, and cut in stone. > We have applied it, and it will be in the first patch release. > > We will also release it as a source code patch immediately > after the R13B release. > > More patches like that! (and earlier)-: > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > As far as I can see the patch has not been applied, or at least it's not present in R13B01. Has it been lost? Regards, Davide -------------- next part -------------- A non-text attachment was scrubbed... Name: erlang-R13B-LDFLAGS.patch Type: application/octet-stream Size: 4225 bytes Desc: not available URL: From aude.quintana@REDACTED Tue Aug 18 11:03:05 2009 From: aude.quintana@REDACTED (Aude Quintana) Date: Tue, 18 Aug 2009 11:03:05 +0200 Subject: Patch for ASN1 Library Message-ID: Hello ! Here is a proposition for a patch for Erlang asn1 library. I hope I didn't reimplement existing functionality, but I didn't manage to do it another way. If the functionality is already implemented, thanks to reply me with some indication on how to proceed using it. So this patch should be used to catch decoding error when the message to decode is incomplete and to return the error {error,incomplete} in this case. The existing behaviour is to throw various hard-to-match errors. This change is necessary when using APER norm. The APER norm makes impossible to foresee the message length and so to foresee if the packet to decode is long enough to contain a full packet. So, when decoding a packet with error, it is impossible to say if it is because the contained message is incomplete or other. With BER encoding it is cheap to read the size of the message, but with PER you have to decode the whole message first, so an in-library solution is better. .diff file attached with the details of my modifications to add the return of the error {error,incomplete} in case of an incomplete message. In the following files : asn1rt_ber_bin.erl asn1rt_ber_bin_v2.erl asn1rt_uper_bin.erl asn1rt_per_bin.erl asn1rt_per_bin_rt2ct.erl the size of the buffer put in argument should be check each time a function wants to match a precise number of bytes (related to an expected size of message). This means * to add a check_bit_size(Buffer,BitSize) or a check_bin_size(Buffer,BinSize) each time we had something like "<> = Buffer", * and to add pattern functions, for example: for getbit(Bin) we must add getbit(<<>>) -> throw({error,incomplete}) and not let the system crashed. Note that check_bit_size/2 et check_bin_size/2 are functions added in each file. Eventually, in the generator file asn1ct_gen.erl, just 2 lines in decode_disp/2 must be added to forward the error {error,incomplete} to the parent module. This seems to be enough to implement the error matching, I tried to test with my data and it works and with bad data, it replies with an error {error,incomplete}. What do you think about it ? Best regards, Aude. -- Aude Quintana -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: patch_lib_asn1.diff Type: text/x-patch Size: 35693 bytes Desc: not available URL: From tomas.abrahamsson@REDACTED Thu Aug 20 07:59:18 2009 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Thu, 20 Aug 2009 07:59:18 +0200 Subject: [erlang-bugs] inets: HTTP client race condition In-Reply-To: <1249037307.13468.50.camel@sredniczarny> References: <1249037307.13468.50.camel@sredniczarny> Message-ID: > Today I have encountered strange problem in simple scenario with inets. > when I'm doing concurrent requests to the same server which responds > slowly. Hi, I too have got bitten by this now, so I extended your test program to not rely on any external web server, to make it a bit easier to trigger and observe the bug. Attached is a program that triggers the bug fairly often in a local environment. Compile it and run it like this: erl -s http_client_race_condition go -s erlang halt Attached is also a patch fixing the problem. The patch is against R13B01/inets-5.1 It seems the problem is that if one calls http:request and the httpc_manager selects a session where there's already a pending request, then the connection handler for that session effectively resets its parser, readying it for the response to the second request, but sometimes there are still some more inbound packets for the response to the first request, and that's when things get confused. I did some research: The bug first appeared in R13A/inets-5.0.13, in which a queue for keep-alive requests seems to have been introduced. In inets-5.0.12 (R12B-5) there was only one pipeline queue, but in 5.0.13 there's also another queue for keep-alive. (I don't know the reason for there being two queues). Incidentally, when fixing the problem, the code for handling the keep-alive queue more and more grew to resemble the code that handles the pipeline queue. Now it looks much like two copies of the same code. So someone more knowledgeable in the art of inets and the pipeline/keep-alive queues might want to refactor the patched code into something more beautiful. As you already observed, creating separate profiles works around the bug. With this patch, it is no longer necessary to create extra profiles if one wants to send http requests concurrently. BRs Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: http_client_race_condition.erl Type: text/x-erlang Size: 4811 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: httpc_handler.erl.patch Type: text/x-patch Size: 3443 bytes Desc: not available URL: From geoff.cant@REDACTED Thu Aug 20 15:49:25 2009 From: geoff.cant@REDACTED (Geoff Cant) Date: Thu, 20 Aug 2009 15:49:25 +0200 Subject: public_key API improvement Message-ID: Hi all, I've just been investigating the new public_key suite for use in one of our projects. It looks very useful, but I have a possible API improvement to share: public_key:pem_to_der/1,2 require that the PEM data come from a file on disk. I'd like to be able to operate on PEM data from non-file sources, so I've extended the API to accept a binary full of PEM data as well. -------------- next part -------------- A non-text attachment was scrubbed... Name: pubkey_api_improvement.diff Type: text/x-patch Size: 3310 bytes Desc: not available URL: -------------- next part -------------- The new tree is available from http://github.com/mfoemmel/erlang-otp/commits/pubkey_api_improvement Cheers, -- Geoff Cant From aude.quintana@REDACTED Tue Aug 25 12:57:58 2009 From: aude.quintana@REDACTED (Aude Quintana) Date: Tue, 25 Aug 2009 12:57:58 +0200 Subject: [Re] Patch for asn1 library Message-ID: Hello ! After some other tests with the patch I made previously on asn1 library I had to add some lines in the patch. (it makes the decoding function to return {error,incomplete} when the buffer passed in argument doesn't contain at least a full message) So, .diff file joined is the new patch and I hope it is complete now. Best Regards, Aude. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: repatch_lib_asn1.diff Type: text/x-patch Size: 37388 bytes Desc: not available URL: From aude.quintana@REDACTED Tue Aug 25 14:28:42 2009 From: aude.quintana@REDACTED (Aude Quintana) Date: Tue, 25 Aug 2009 14:28:42 +0200 Subject: Patch for asn1 library -> UPER Message-ID: Hello ! Here is another patch for Erlang asn1 library -> UPER The UPER encoding function for OCTETSTRING doesn't accept binaries in argument. It's strange because * BER function accept them * The encoding function of UPER transforms the list passed in argument in a binary. * it would be needlessly wastefull to wrap our data inside a list when it is already a binary So it should directly accept binaries in argument too. .diff file attached is a patch which adds this functionality. It just modifies encode_octet_string function in * asn1rt_uper_bin.erl There is the same problem with APER (files asn1rt_per_bin.erl and asn1rt_per_bin_rt2ct.erl) but it would take too much time to change it because it's more complicated in these files. Best regards, Aude. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: patch_binlist.diff Type: text/x-patch Size: 1131 bytes Desc: not available URL: From vinoski@REDACTED Wed Aug 26 16:49:22 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 26 Aug 2009 10:49:22 -0400 Subject: patch for sys:get_status to call format_status/2 In-Reply-To: <65b2728e0908260612yfd6c1c9ke8f6f3d9a5b652f8@mail.gmail.com> References: <65b2728e0908260612yfd6c1c9ke8f6f3d9a5b652f8@mail.gmail.com> Message-ID: <65b2728e0908260749k2e5b2ce3g17a4882feff2cd82@mail.gmail.com> Back in March of this year the following thread appeared in erlang-questions regarding the undocumented format_status/2 function that apparently used to be called by sys:get_status if provided by a behavior module: I searched back all the way to version R6B but found no code that made that call except within SASL's si_sasl_supp module, yet I see that various modules such as gen_server, gen_event, gen_fsm, and os_mon's disksup and memsup each provide a format_status/2 function, plus the generic behaviors also check to see if modules implementing them also provide a format_status/2 function, calling it if they do and thus allowing you to format how sys:get_status prints out your gen_server, gen_fsm, etc. state. Seems like a shame to let that code go to waste! :-) I agree with the originator of that thread that being allowed to provide a special format_status/2 when implementing a behavior can be very handy, so below find a simple patch for R13B01 that implements it. --steve --- lib/stdlib/src/sys.erl~ 2009-03-12 08:18:15.000000000 -0400 +++ lib/stdlib/src/sys.erl 2009-08-26 01:25:13.000000000 -0400 @@ -245,8 +245,17 @@ {SysState, {error, {unknown_system_msg, Other}}, Debug, Misc}. get_status(SysState, Parent, Mod, Debug, Misc) -> + PDict = get(), + FmtMisc = + case erlang:function_exported(Mod, format_status, 2) of + true -> + FmtArgs = [PDict, SysState, Parent, Debug, Misc], + Mod:format_status(normal, FmtArgs); + _ -> + Misc + end, {status, self(), {module, Mod}, - [get(), SysState, Parent, Debug, Misc]}. + [PDict, SysState, Parent, Debug, FmtMisc]}. %%----------------------------------------------------------------- %% These are the system debug commands. From vinoski@REDACTED Wed Aug 26 20:59:06 2009 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 26 Aug 2009 14:59:06 -0400 Subject: Fwd: xmerl ftp URI parsing patch In-Reply-To: <65b2728e0907152154l6679ca16m114e376ea0d07e63@mail.gmail.com> References: <65b2728e0907152154l6679ca16m114e376ea0d07e63@mail.gmail.com> Message-ID: <65b2728e0908261159v73ad30cal5a73c784b618a08a@mail.gmail.com> I haven't seen any replies to the patch below, and given that I sent it during the height of summer holidays and also given recent problems I've had sending to these mailing lists, I thought I'd resend just in case. --steve ---------- Forwarded message ---------- From: Steve Vinoski Date: Thu, Jul 16, 2009 at 12:54 AM Subject: xmerl ftp URI parsing patch To: erlang-bugs@REDACTED The xmerl_uri:parse function can't handle FTP URIs containing username and password. Also, the default FTP port constant in xmerl_uri.erl is wrong. Here's a patch against R13B01 that I believe remedies these issues. I tried to write it to match the style of the other code in that file. I manually ran a few of the W3C XML schema URI test FTP URIs against this and they all looked good. Note, though, that this patch doesn't handle encoded :, @, or / characters appearing within the username or password fields as required by RFC 1738. Note also that I left the "FIXME" comment in the code, but with these changes you might want to consider removing it. These changes also work for R12B-5 but the line offsets are different. --steve --- otp_src_R13B01/lib/xmerl/src/xmerl_uri.erl 2009-03-12 08:27:06.000000000 -0400 +++ xmerl_uri.erl 2009-07-16 00:44:27.000000000 -0400 @@ -120,7 +120,7 @@ %%% ............................................................................ %%% FIXME!!! This is just a quick hack that doesn't work! --define(FTP_DEFAULT_PORT, 80). +-define(FTP_DEFAULT_PORT, 21). %%% FTP (Source RFC 2396, RFC 1738, RFC 959) %%% Note: This BNF has been modified to better fit with RFC 2396 @@ -155,9 +155,21 @@ end. ftp_userinfo(C0) -> - User="", - Password="", - {C0,{User,Password}}. + ftp_userinfo(C0,0,[],[],[]). +ftp_userinfo([],_,Acc,User,Passwd) -> + {lists:reverse(Acc),{User,Passwd}}; +ftp_userinfo(C0=[$/|_],_,Acc,User,Passwd) -> + {lists:reverse(Acc)++C0,{User,Passwd}}; +ftp_userinfo([$@|C0],1,Acc,User,[]) -> + {C0,{User,lists:reverse(Acc)}}; +ftp_userinfo([$@|C0],0,Acc,[],[]) -> + {C0,{lists:reverse(Acc),""}}; +ftp_userinfo([$:|_],0,[],[],[]) -> + {error,no_user}; +ftp_userinfo([$:|C0],0,Acc,[],[]) -> + ftp_userinfo(C0,1,[],lists:reverse(Acc),[]); +ftp_userinfo([C|C0],Stage,Acc,User,Passwd) -> + ftp_userinfo(C0,Stage,[C|Acc],User,Passwd). %%% ......................................................................... From dgud@REDACTED Fri Aug 28 08:14:05 2009 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 28 Aug 2009 08:14:05 +0200 Subject: [erlang-patches] public_key API improvement In-Reply-To: References: Message-ID: <4A9775AD.8040609@erix.ericsson.se> Thanks, added. /Dan Geoff Cant wrote: > Hi all, I've just been investigating the new public_key suite for use in > one of our projects. It looks very useful, but I have a possible API > improvement to share: > > public_key:pem_to_der/1,2 require that the PEM data come from a file on > disk. I'd like to be able to operate on PEM data from non-file sources, > so I've extended the API to accept a binary full of PEM data as well. > > > > ------------------------------------------------------------------------ > > > The new tree is available from > http://github.com/mfoemmel/erlang-otp/commits/pubkey_api_improvement > > Cheers, > > > ------------------------------------------------------------------------ > > > ________________________________________________________________ > erlang-patches mailing list. See http://www.erlang.org/faq.html > erlang-patches (at) erlang.org From lars@REDACTED Fri Aug 28 11:54:52 2009 From: lars@REDACTED (Lars Thorsen) Date: Fri, 28 Aug 2009 11:54:52 +0200 Subject: [erlang-patches] Fwd: xmerl ftp URI parsing patch In-Reply-To: <4A963582.2080901@erix.ericsson.se> References: <4A963582.2080901@erix.ericsson.se> Message-ID: <4A97A96C.4020402@erix.ericsson.se> Thanks for the patch. We'll include it in R13B02. Best regards, /Lars , Erlang/OTP Ericsson > > > -------- Original Message -------- > Subject: [erlang-patches] Fwd: xmerl ftp URI parsing patch > Date: Wed, 26 Aug 2009 14:59:06 -0400 > From: Steve Vinoski > To: erlang-patches@REDACTED > References: <65b2728e0907152154l6679ca16m114e376ea0d07e63@REDACTED> > > I haven't seen any replies to the patch below, and given that I sent it > during the height of summer holidays and also given recent problems I've > had > sending to these mailing lists, I thought I'd resend just in case. > --steve > > ---------- Forwarded message ---------- > From: Steve Vinoski > Date: Thu, Jul 16, 2009 at 12:54 AM > Subject: xmerl ftp URI parsing patch > To: erlang-bugs@REDACTED > > > The xmerl_uri:parse function can't handle FTP URIs containing username > and password. Also, the default FTP port constant in xmerl_uri.erl is > wrong. Here's a patch against R13B01 that I believe remedies these > issues. I tried to write it to match the style of the other code in > that file. > > I manually ran a few of the W3C XML schema URI test FTP URIs against > this and they all looked good. Note, though, that this patch doesn't > handle encoded :, @, or / characters appearing within the username or > password fields as required by RFC 1738. Note also that I left the > "FIXME" comment in the code, but with these changes you might want to > consider removing it. > > These changes also work for R12B-5 but the line offsets are different. > > --steve > > --- otp_src_R13B01/lib/xmerl/src/xmerl_uri.erl 2009-03-12 > 08:27:06.000000000 -0400 > +++ xmerl_uri.erl 2009-07-16 00:44:27.000000000 -0400 > @@ -120,7 +120,7 @@ > > %%% > ............................................................................ > > %%% FIXME!!! This is just a quick hack that doesn't work! > --define(FTP_DEFAULT_PORT, 80). > +-define(FTP_DEFAULT_PORT, 21). > > %%% FTP (Source RFC 2396, RFC 1738, RFC 959) > %%% Note: This BNF has been modified to better fit with RFC 2396 > @@ -155,9 +155,21 @@ > end. > > ftp_userinfo(C0) -> > - User="", > - Password="", > - {C0,{User,Password}}. > + ftp_userinfo(C0,0,[],[],[]). > +ftp_userinfo([],_,Acc,User,Passwd) -> > + {lists:reverse(Acc),{User,Passwd}}; > +ftp_userinfo(C0=[$/|_],_,Acc,User,Passwd) -> > + {lists:reverse(Acc)++C0,{User,Passwd}}; > +ftp_userinfo([$@|C0],1,Acc,User,[]) -> > + {C0,{User,lists:reverse(Acc)}}; > +ftp_userinfo([$@|C0],0,Acc,[],[]) -> > + {C0,{lists:reverse(Acc),""}}; > +ftp_userinfo([$:|_],0,[],[],[]) -> > + {error,no_user}; > +ftp_userinfo([$:|C0],0,Acc,[],[]) -> > + ftp_userinfo(C0,1,[],lists:reverse(Acc),[]); > +ftp_userinfo([C|C0],Stage,Acc,User,Passwd) -> > + ftp_userinfo(C0,Stage,[C|Acc],User,Passwd). > > > %%% > ......................................................................... > From davidepesa@REDACTED Fri Aug 28 13:02:06 2009 From: davidepesa@REDACTED (Davide Pesavento) Date: Fri, 28 Aug 2009 13:02:06 +0200 Subject: Build system ignores LDFLAGS In-Reply-To: <2da21fe50908171403n58809a00ob0eb59d5866a3007@mail.gmail.com> References: <2da21fe50908161305n6b3ce3cam3283ed47fadf91e6@mail.gmail.com> <2da21fe50908171403n58809a00ob0eb59d5866a3007@mail.gmail.com> Message-ID: <2da21fe50908280402i5b80ee78h4d7920bf26491be4@mail.gmail.com> On Mon, Apr 20, 2009 at 14:30, Raimo Niskanen wrote: > On Fri, Apr 17, 2009 at 11:34:49PM +0200, Davide Pesavento wrote: >> On Sat, Apr 4, 2009 at 13:05, Davide Pesavento wrote: >> > On Tue, Mar 31, 2009 at 22:13, Davide Pesavento wrote: >> >> Hello, >> >> >> >> Erlang's build system seems to ignore custom LDFLAGS. This is the QA >> >> warning shown by Portage (Gentoo Linux's package manager) after >> >> building R12B-5 with LDFLAGS="-Wl,-O1 -Wl,--as-needed >> >> -Wl,--hash-style=gnu" >> >> >> >> QA Notice: Files built without respecting LDFLAGS have been detected >> >> ?Please include the following list of files in your report: >> >> /usr/lib64/erlang/lib/runtime_tools-1.7.3/priv/lib/trace_file_drv.so >> >> /usr/lib64/erlang/lib/runtime_tools-1.7.3/priv/lib/trace_ip_drv.so >> >> /usr/lib64/erlang/lib/megaco-3.9.1.1/priv/lib/megaco_flex_scanner_drv_mt.so >> >> /usr/lib64/erlang/lib/megaco-3.9.1.1/priv/lib/megaco_flex_scanner_drv.so >> >> /usr/lib64/erlang/lib/erl_interface-3.5.9/bin/erl_call >> >> /usr/lib64/erlang/lib/ssl-3.10/priv/bin/ssl_esock >> >> /usr/lib64/erlang/lib/asn1-1.6.2/priv/lib/asn1_erl_drv.so >> >> /usr/lib64/erlang/lib/crypto-1.5.3/priv/lib/crypto_drv.so >> >> /usr/lib64/erlang/lib/common_test-1.3.4/priv/lib/erl_rx_driver.so >> >> /usr/lib64/erlang/erts-5.6.5/bin/child_setup >> >> >> >> Before digging into the Makefiles, I'd like to know if you already >> >> have some ideas about the root cause of this issue. >> >> >> > >> > This bug is still present in R13A. >> > >> > ?* QA Notice: Files built without respecting LDFLAGS have been detected >> > ?* ?Please include the following list of files in your report: >> > ?* /usr/lib64/erlang/lib/runtime_tools-1.8/priv/lib/trace_file_drv.so >> > ?* /usr/lib64/erlang/lib/runtime_tools-1.8/priv/lib/trace_ip_drv.so >> > ?* /usr/lib64/erlang/lib/erl_interface-3.6/bin/erl_call >> > ?* /usr/lib64/erlang/lib/crypto-1.6/priv/lib/crypto_drv.so >> > ?* /usr/lib64/erlang/lib/megaco-3.10.0.1/priv/lib/megaco_flex_scanner_drv_mt.so >> > ?* /usr/lib64/erlang/lib/megaco-3.10.0.1/priv/lib/megaco_flex_scanner_drv.so >> > ?* /usr/lib64/erlang/lib/ssl-3.10.1/priv/bin/ssl_esock >> > ?* /usr/lib64/erlang/lib/asn1-1.6.8/priv/lib/asn1_erl_drv.so >> > ?* /usr/lib64/erlang/erts-5.7/bin/child_setup >> > >> > Best regards, >> > Davide >> > >> >> The attached patch fixes the problem for me: LDFLAGS are now respected. >> >> Regards, >> Davide > > The patch looks great! It is unfortunately a few days late to make > it into R13B. Now the source tree is labeled, and cut in stone. > We have applied it, and it will be in the first patch release. > > We will also release it as a source code patch immediately > after the R13B release. > > More patches like that! (and earlier)-: > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > As far as I can see the patch has not been applied, or at least it's not present in R13B01. Has it been lost? Regards, Davide -------------- next part -------------- A non-text attachment was scrubbed... Name: erlang-R13B-LDFLAGS.patch Type: application/octet-stream Size: 4225 bytes Desc: not available URL: