From daniel.goertzen@REDACTED Wed Sep 3 23:00:18 2014 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 3 Sep 2014 16:00:18 -0500 Subject: [erlang-bugs] x509 certificate decoded string representation is wrong In-Reply-To: <53DA57D2.2010809@ericsson.com> References: <53DA57D2.2010809@ericsson.com> Message-ID: Now its my turn to apologize for the late response. :) I ran the tests again on 17.0 and the same issues exist. Here is my cert generator crashing on a unicode string: 15> api_app:generate_cert_with_key(1024). ** exception error: no match of right hand side value {error,{asn1,badarg}} in function pubkey_cert_records:transform/2 (pubkey_cert_records.erl, line 59) in call from lists:map/2 (lists.erl, line 1237) in call from pubkey_cert_records:transform/2 (pubkey_cert_records.erl, line 91) in call from pubkey_cert_records:encode_tbs/1 (pubkey_cert_records.erl, line 306) in call from public_key:pkix_encode/3 (public_key.erl, line 268) in call from public_key:pkix_sign/2 (public_key.erl, line 472) in call from api_app:generate_cert_with_key/1 (src/api_app.erl, line 128) The part of the cert with the unicode string was... Subject = {rdnSequence, [ [#'AttributeTypeAndValue'{ type = ?'id-at-commonName', value = {utf8String, [16#4e09|" string starting with a chinese symbol"]} %% value = {utf8String, "embedded self-signed cert"} }] ]}, It works fine when I swap in the non-unicode string. Here is a certificate and it's decode showing how the text is presented as binaries instead of strings (lists). goertzen@REDACTED ~/test $ cat test.cert -----BEGIN CERTIFICATE----- MIIDXTCCAkWgAwIBAgIJALPOPyhAojyyMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX aWRnaXRzIFB0eSBMdGQwHhcNMTQwNjA0MTQyNjEyWhcNMTcwNjAzMTQyNjEyWjBF MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB CgKCAQEA5M68fzYaR1Ef7DT1ooDw/QhwK2euedf7Lj1BugVawuhSGpHgsrVUorzP UN4GXXWoavD0I9KMyIhY8Wzh5iI9xwjZ3RtQGLcSdWAiQME/7dPfs94MzcSKATgS ZePunlOWFKxP9ie12GKKyvo4PtwRRZ/m50967DbYH8d+Tg+ASTWYA8EHrpBrlvox dA9e5xkiLwt15+SnDhiR06czy7XQ/+4oJeoIwB66iah/LVe3PXnJU/+qYSqCSgYv q41L+0FSYpTBqRbJ72WxT7l2lj0IsKHkT6ywuvwt6MCw5g96Wgvsyn71b9EUKA6K lTllaA96iqVHVouvX58t4D4Zt8Ic+QIDAQABo1AwTjAdBgNVHQ4EFgQUeO5XkCL9 omKoM6uKWezYWCKcwoIwHwYDVR0jBBgwFoAUeO5XkCL9omKoM6uKWezYWCKcwoIw DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAkFPuHkWUr3ZE0goNzs6R uRnNZESffEFXtk8FMQXUV+9i8dJbE0ElK7MwVLbc4VS/nXoBVQbicSTOPq2LZe0g b72YcrtQlMoWdwNCibjnuxdI0o76nfWShxvy6K1sJ0qQ/T3fsfvGgKn8qGDCAGmY 2PJaaXvpqSy3wIPIFxnhCGiuCrcVudZkJwVWnG27jYd5IE4i9st5oJnatMS0rCeG InjWWg6aAo5hsdfgug0mMyX+87EixoGfUNm59TWWViWcFuLUksAKECmzseJZLBnT 1+bHXIGyV/NxZW90R8NC5ObBkP924kCdAGKPWVW6XruMCFlrmeU+P9RKhiwf/FX4 zg== -----END CERTIFICATE----- goertzen@REDACTED ~/test $ erl Erlang/OTP 17 [erts-6.0.1] [source-deacab9] [64-bit] [smp:3:3] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.0.1 (abort with ^G) 1> {ok, PemBin} = file:read_file("test.cert"). {ok,<<"-----BEGIN CERTIFICATE-----\nMIIDXTCCAkWgAwIBAgIJALPOPyhAojyyMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV\nBAYTAkFVMRMwEQY"...>>} 2> [{'Certificate',Cert,not_encrypted}] = public_key:pem_decode(PemBin). [{'Certificate',<<48,130,3,93,48,130,2,69,160,3,2,1,2,2,9, 0,179,206,63,40,64,162,60,178,48,13,...>>, not_encrypted}] 3> OTPCert = public_key:pkix_decode_cert(Cert, otp). {'OTPCertificate',{'OTPTBSCertificate',v3, 12956362620107111602, {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, {rdnSequence,[[{'AttributeTypeAndValue',{2,5,4,6},"AU"}], [{'AttributeTypeAndValue',{2,5,4,8}, *{utf8String,<<"Some-State">>*}}], [{'AttributeTypeAndValue',{2,5,4,10}, *{utf8String,<<"Internet Widgits Pty Ltd">>}*}]]}, {'Validity',{utcTime,"140604142612Z"}, {utcTime,"170603142612Z"}}, {rdnSequence,[[{'AttributeTypeAndValue',{2,5,4,6},"AU"}], [{'AttributeTypeAndValue',{2,5,4,8}, {utf8String,<<"Some-State">>}}], [{'AttributeTypeAndValue',{2,5,4,10}, {utf8String,<<"Internet Widgits Pty Ltd">>}}]]}, {'OTPSubjectPublicKeyInfo',{'PublicKeyAlgorithm',{1,2,840, 113549,1,1,1}, 'NULL'}, {'RSAPublicKey',28884279009285790301669924467575946032489944489699263464818187209641451094053086029317223819905552232804663206681799701028265354352239301173419976030286993076554223850305834956052085323986279172838215343728630359816997644527827805951010425788227425209795979178217250409900809512057807966976585078052255837974497261381284713573904257439066194709912683930375930487604830660421775765930552658716300974673840116044766673767999239289293469792318108847067157652307614870531765029586512631237561271128048065184664689086457528061690000433623928374995254617336829376087731238998361247235770397685269871591203885794507056356601, 65537}}, asn1_NOVALUE,asn1_NOVALUE, [{'Extension',{2,5,29,14}, false, [120,238,87,144,34,253,162,98,168,51,171|...]}, {'Extension',{2,5,29,35}, false, {'AuthorityKeyIdentifier',[120,238,87,144,34,253,162,98|...], asn1_NOVALUE,asn1_NOVALUE}}, {'Extension',{2,5,29,19}, false, {'BasicConstraints',true,asn1_NOVALUE}}]}, {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, {0, <<144,83,238,30,69,148,175,118,68,210,10,13,206,206,145, 185,25,205,100,68,159,124,65,...>>}} 4> Dan. On Thu, Jul 31, 2014 at 9:50 AM, Ingela Anderton Andin < Ingela.Anderton.Andin@REDACTED> wrote: > Hi! > > Sorry for the late answer. Was this pre 17.0 or 17.0 ? There was a > unicode fix in 17.0 the accidentally seems to have been lost in the release > notes. If you still have problems could you please send us a sample cert > that fails to speed up the process. > > Regards Ingela Erlang/OTP team - Ericsson AB > > > > > > On 06/12/2014 10:57 PM, Daniel Goertzen wrote: > >> In the public_key application, decoding of attributes in x509 >> certificates does not always decode to a string as indicated in the >> documentation. The documentation says that the value of commonName (and >> several other attributes) should be: >> >> special_string() = {teletexString, string()} | {printableString, >> string()} | {universalString, string()} | {utf8String, string()} | >> {bmpString, string()} >> >> ... however when I decode a cert I see a utf8String coming out as a >> binary instead of a string().... >> >> [{'AttributeTypeAndValue', >> {2,5,4,3}, >> {utf8String,<<"Daniel Goertzen">>}}], >> >> ... and typer shows several other non-string representations (unicode >> characters represented by 4-tuples): >> >> -spec dec_X520CommonName(_) -> >> {'bmpString',[byte() | {byte(),byte(),byte(),byte()}] | >> {byte(),binary()}} | >> {'printableString',[byte() | {byte(),byte(),byte(),byte()}] | >> {byte(),binary()}} | >> {'teletexString',[byte() | {byte(),byte(),byte(),byte()}] | >> {byte(),binary()}} | >> {'universalString',[byte() | {byte(),byte(),byte(),byte()}] | >> {byte(),binary()}} | >> {'utf8String',_}. >> >> >> >> Also, encoding does not accept unicode strings (list of chars). The >> example below crashes. >> >> Subject = {rdnSequence, [ >> [#'AttributeTypeAndValue'{ >> type = ?'id-at-commonName', >> value = {utf8String, [16#4e09|" string starting with a >> chinese symbol"]} >> }] >> ]}, >> >> >> >> I assume that the documentation is right and the decoded representation >> should be "list of characters". Instead internal representations are >> coming through. >> >> >> _______________________________________________ >> erlang-bugs mailing list >> erlang-bugs@REDACTED >> http://erlang.org/mailman/listinfo/erlang-bugs >> >> > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Wed Sep 3 23:32:32 2014 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Wed, 03 Sep 2014 23:32:32 +0200 Subject: [erlang-bugs] x509 certificate decoded string representation is wrong In-Reply-To: References: <53DA57D2.2010809@ericsson.com> Message-ID: <540788F0.5080605@wanadoo.fr> Hello, I confirm it is not seen as PRINTABLESTRING but as default VALUE, using swab. Regards 1> {ok, Pem} = file:read_file("/tmp/certificate.pem"). {ok,<<"-----BEGIN CERTIFICATE-----\nMIIDXTCCAkWgAwIBAgIJALPOPyhAojyyMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV\nBAYTAkFVMRMwEQY"...>>} 2> swab:sync([{convert,der},{debug,asn1_pp}],Pem). <0.32.0> : {debug,asn1_pp} => SEQUENCE . SEQUENCE . . CONSTRUCTOR . . . INTEGER : 10#2 (16#2) . . . END . . INTEGER : 10#12956362620107111602 (16#B3CE3F2840A23CB2) . . SEQUENCE . . . OBJECT : {1,2,840,113549,1,1,11} . . . NULL : <<>> . . . END . . SEQUENCE . . . SET . . . . SEQUENCE . . . . . OBJECT : {2,5,4,6} . . . . . PRINTABLESTRING : AU . . . . . END . . . . END . . . SET . . . . SEQUENCE . . . . . OBJECT : {2,5,4,8} . . . . . VALUE : Some-State . . . . . END . . . . END . . . SET . . . . SEQUENCE . . . . . OBJECT : {2,5,4,10} . . . . . VALUE : Internet Widgits Pty Ltd . . . . . END . . . . END . . . END . . SEQUENCE . . . UTCTIME : 140604142612Z (2014-06-04 16:26:12 UTC+2) . . . UTCTIME : 170603142612Z (2017-06-03 16:26:12 UTC+2) . . . END . . SEQUENCE . . . SET . . . . SEQUENCE . . . . . OBJECT : {2,5,4,6} . . . . . PRINTABLESTRING : AU . . . . . END . . . . END . . . SET . . . . SEQUENCE . . . . . OBJECT : {2,5,4,8} . . . . . VALUE : Some-State . . . . . END . . . . END . . . SET . . . . SEQUENCE . . . . . OBJECT : {2,5,4,10} . . . . . VALUE : Internet Widgits Pty Ltd . . . . . END . . . . END . . . END . . SEQUENCE . . . SEQUENCE . . . . OBJECT : {1,2,840,113549,1,1,1} . . . . NULL : <<>> . . . . END . . . BIT STRING : 00000000 00 30 82 01 0a 02 82 01 01 00 e4 ce bc 7f 36 1a |.0...........^6.| 00000010 47 51 1f ec 34 f5 a2 80 f0 fd 08 70 2b 67 ae 79 |GQ..4......p+g.y| 00000020 d7 fb 2e 3d 41 ba 05 5a c2 e8 52 1a 91 e0 b2 b5 |...=A..Z..R.....| 00000030 54 a2 bc cf 50 de 06 5d 75 a8 6a f0 f4 23 d2 8c |T...P..]u.j..#..| 00000040 c8 88 58 f1 6c e1 e6 22 3d c7 08 d9 dd 1b 50 18 |..X.l.."=.....P.| 00000050 b7 12 75 60 22 40 c1 3f ed d3 df b3 de 0c cd c4 |..u`"@.?........| 00000060 8a 01 38 12 65 e3 ee 9e 53 96 14 ac 4f f6 27 b5 |..8.e...S...O.'.| 00000070 d8 62 8a ca fa 38 3e dc 11 45 9f e6 e7 4f 7a ec |.b...8>..E...Oz.| 00000080 36 d8 1f c7 7e 4e 0f 80 49 35 98 03 c1 07 ae 90 |6...~N..I5......| 00000090 6b 96 fa 31 74 0f 5e e7 19 22 2f 0b 75 e7 e4 a7 |k..1t.^.."/.u...| 000000a0 0e 18 91 d3 a7 33 cb b5 d0 ff ee 28 25 ea 08 c0 |.....3.....(%...| 000000b0 1e ba 89 a8 7f 2d 57 b7 3d 79 c9 53 ff aa 61 2a |....^-W.=y.S..a*| 000000c0 82 4a 06 2f ab 8d 4b fb 41 52 62 94 c1 a9 16 c9 |.J./..K.ARb.....| 000000d0 ef 65 b1 4f b9 76 96 3d 08 b0 a1 e4 4f ac b0 ba |.e.O.v.=....O...| 000000e0 fc 2d e8 c0 b0 e6 0f 7a 5a 0b ec ca 7e f5 6f d1 |.-.....zZ...~.o.| 000000f0 14 28 0e 8a 95 39 65 68 0f 7a 8a a5 47 56 8b af |.(...9eh.z..GV..| 00000100 5f 9f 2d e0 3e 19 b7 c2 1c f9 02 03 01 00 01 |_.-.>..........| . . . END . . CONSTRUCTOR . . . SEQUENCE . . . . SEQUENCE . . . . . OBJECT : {2,5,29,14} . . . . . OCTET STRING : 00000000 04 14 78 ee 57 90 22 fd a2 62 a8 33 ab 8a 59 ec |..x.W."..b.3..Y.| 00000010 d8 58 22 9c c2 82 |.X"...| . . . . . END . . . . SEQUENCE . . . . . OBJECT : {2,5,29,35} . . . . . OCTET STRING : 00000000 30 16 80 14 78 ee 57 90 22 fd a2 62 a8 33 ab 8a |0...x.W."..b.3..| 00000010 59 ec d8 58 22 9c c2 82 |Y..X"...| . . . . . END . . . . SEQUENCE . . . . . OBJECT : {2,5,29,19} . . . . . OCTET STRING : 00000000 30 03 01 01 ff |0....| . . . . . END . . . . END . . . END . . END . SEQUENCE . . OBJECT : {1,2,840,113549,1,1,11} . . NULL : <<>> . . END . BIT STRING : 00000000 00 90 53 ee 1e 45 94 af 76 44 d2 0a 0d ce ce 91 |..S..E..vD......| 00000010 b9 19 cd 64 44 9f 7c 41 57 b6 4f 05 31 05 d4 57 |...dD.|AW.O.1..W| 00000020 ef 62 f1 d2 5b 13 41 25 2b b3 30 54 b6 dc e1 54 |.b..[.A%+.0T...T| 00000030 bf 9d 7a 01 55 06 e2 71 24 ce 3e ad 8b 65 ed 20 |..z.U..q$.>..e. | 00000040 6f bd 98 72 bb 50 94 ca 16 77 03 42 89 b8 e7 bb |o..r.P...w.B....| 00000050 17 48 d2 8e fa 9d f5 92 87 1b f2 e8 ad 6c 27 4a |.H...........l'J| 00000060 90 fd 3d df b1 fb c6 80 a9 fc a8 60 c2 00 69 98 |..=........`..i.| 00000070 d8 f2 5a 69 7b e9 a9 2c b7 c0 83 c8 17 19 e1 08 |..Zi{..,........| 00000080 68 ae 0a b7 15 b9 d6 64 27 05 56 9c 6d bb 8d 87 |h......d'.V.m...| 00000090 79 20 4e 22 f6 cb 79 a0 99 da b4 c4 b4 ac 27 86 |y N"..y.......'.| 000000a0 22 78 d6 5a 0e 9a 02 8e 61 b1 d7 e0 ba 0d 26 33 |"x.Z....a.....&3| 000000b0 25 fe f3 b1 22 c6 81 9f 50 d9 b9 f5 35 96 56 25 |%..."...P...5.V%| 000000c0 9c 16 e2 d4 92 c0 0a 10 29 b3 b1 e2 59 2c 19 d3 |........)...Y,..| 000000d0 d7 e6 c7 5c 81 b2 57 f3 71 65 6f 74 47 c3 42 e4 |...\..W.qeotG.B.| 000000e0 e6 c1 90 ff 76 e2 40 9d 00 62 8f 59 55 ba 5e bb |....v.@REDACTED^.| 000000f0 8c 08 59 6b 99 e5 3e 3f d4 4a 86 2c 1f fc 55 f8 |..Yk..>?.J.,..U.| 00000100 ce |.| . END END {ok,<<48,130,3,93,48,130,2,69,160,3,2,1,2,2,9,0,179,206, 63,40,64,162,60,178,48,13,6,...>>} Le 03/09/2014 23:00, Daniel Goertzen a ?crit : > Now its my turn to apologize for the late response. :) > > I ran the tests again on 17.0 and the same issues exist. > > > Here is my cert generator crashing on a unicode string: > > 15> api_app:generate_cert_with_key(1024). > ** exception error: no match of right hand side value {error,{asn1,badarg}} > in function pubkey_cert_records:transform/2 > (pubkey_cert_records.erl, line 59) > in call from lists:map/2 (lists.erl, line 1237) > in call from pubkey_cert_records:transform/2 > (pubkey_cert_records.erl, line 91) > in call from pubkey_cert_records:encode_tbs/1 > (pubkey_cert_records.erl, line 306) > in call from public_key:pkix_encode/3 (public_key.erl, line 268) > in call from public_key:pkix_sign/2 (public_key.erl, line 472) > in call from api_app:generate_cert_with_key/1 (src/api_app.erl, > line 128) > > > The part of the cert with the unicode string was... > > Subject = {rdnSequence, [ > [#'AttributeTypeAndValue'{ > type = ?'id-at-commonName', > value = {utf8String, [16#4e09|" string starting with a > chinese symbol"]} > %% value = {utf8String, "embedded self-signed cert"} > }] > ]}, > > It works fine when I swap in the non-unicode string. > > > > > > Here is a certificate and it's decode showing how the text is presented > as binaries instead of strings (lists). > > > > goertzen@REDACTED ~/test > $ cat test.cert > -----BEGIN CERTIFICATE----- > MIIDXTCCAkWgAwIBAgIJALPOPyhAojyyMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV > BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX > aWRnaXRzIFB0eSBMdGQwHhcNMTQwNjA0MTQyNjEyWhcNMTcwNjAzMTQyNjEyWjBF > MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 > ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB > CgKCAQEA5M68fzYaR1Ef7DT1ooDw/QhwK2euedf7Lj1BugVawuhSGpHgsrVUorzP > UN4GXXWoavD0I9KMyIhY8Wzh5iI9xwjZ3RtQGLcSdWAiQME/7dPfs94MzcSKATgS > ZePunlOWFKxP9ie12GKKyvo4PtwRRZ/m50967DbYH8d+Tg+ASTWYA8EHrpBrlvox > dA9e5xkiLwt15+SnDhiR06czy7XQ/+4oJeoIwB66iah/LVe3PXnJU/+qYSqCSgYv > q41L+0FSYpTBqRbJ72WxT7l2lj0IsKHkT6ywuvwt6MCw5g96Wgvsyn71b9EUKA6K > lTllaA96iqVHVouvX58t4D4Zt8Ic+QIDAQABo1AwTjAdBgNVHQ4EFgQUeO5XkCL9 > omKoM6uKWezYWCKcwoIwHwYDVR0jBBgwFoAUeO5XkCL9omKoM6uKWezYWCKcwoIw > DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAkFPuHkWUr3ZE0goNzs6R > uRnNZESffEFXtk8FMQXUV+9i8dJbE0ElK7MwVLbc4VS/nXoBVQbicSTOPq2LZe0g > b72YcrtQlMoWdwNCibjnuxdI0o76nfWShxvy6K1sJ0qQ/T3fsfvGgKn8qGDCAGmY > 2PJaaXvpqSy3wIPIFxnhCGiuCrcVudZkJwVWnG27jYd5IE4i9st5oJnatMS0rCeG > InjWWg6aAo5hsdfgug0mMyX+87EixoGfUNm59TWWViWcFuLUksAKECmzseJZLBnT > 1+bHXIGyV/NxZW90R8NC5ObBkP924kCdAGKPWVW6XruMCFlrmeU+P9RKhiwf/FX4 > zg== > -----END CERTIFICATE----- > goertzen@REDACTED ~/test > $ erl > Erlang/OTP 17 [erts-6.0.1] [source-deacab9] [64-bit] [smp:3:3] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V6.0.1 (abort with ^G) > 1> {ok, PemBin} = file:read_file("test.cert"). > {ok,<<"-----BEGIN > CERTIFICATE-----\nMIIDXTCCAkWgAwIBAgIJALPOPyhAojyyMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV\nBAYTAkFVMRMwEQY"...>>} > 2> [{'Certificate',Cert,not_encrypted}] = public_key:pem_decode(PemBin). > [{'Certificate',<<48,130,3,93,48,130,2,69,160,3,2,1,2,2,9, > 0,179,206,63,40,64,162,60,178,48,13,...>>, > not_encrypted}] > 3> OTPCert = public_key:pkix_decode_cert(Cert, otp). > {'OTPCertificate',{'OTPTBSCertificate',v3, > 12956362620107111602, > > {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, > > {rdnSequence,[[{'AttributeTypeAndValue',{2,5,4,6},"AU"}], > > [{'AttributeTypeAndValue',{2,5,4,8}, > *{utf8String,<<"Some-State">>*}}], > > [{'AttributeTypeAndValue',{2,5,4,10}, > *{utf8String,<<"Internet Widgits Pty Ltd">>}*}]]}, > > {'Validity',{utcTime,"140604142612Z"}, > > {utcTime,"170603142612Z"}}, > > {rdnSequence,[[{'AttributeTypeAndValue',{2,5,4,6},"AU"}], > > [{'AttributeTypeAndValue',{2,5,4,8}, > {utf8String,<<"Some-State">>}}], > > [{'AttributeTypeAndValue',{2,5,4,10}, > {utf8String,<<"Internet Widgits Pty Ltd">>}}]]}, > > {'OTPSubjectPublicKeyInfo',{'PublicKeyAlgorithm',{1,2,840, > 113549,1,1,1}, > 'NULL'}, > > {'RSAPublicKey',28884279009285790301669924467575946032489944489699263464818187209641451094053086029317223819905552232804663206681799701028265354352239301173419976030286993076554223850305834956052085323986279172838215343728630359816997644527827805951010425788227425209795979178217250409900809512057807966976585078052255837974497261381284713573904257439066194709912683930375930487604830660421775765930552658716300974673840116044766673767999239289293469792318108847067157652307614870531765029586512631237561271128048065184664689086457528061690000433623928374995254617336829376087731238998361247235770397685269871591203885794507056356601, > 65537}}, > asn1_NOVALUE,asn1_NOVALUE, > [{'Extension',{2,5,29,14}, > false, > > [120,238,87,144,34,253,162,98,168,51,171|...]}, > {'Extension',{2,5,29,35}, > false, > > {'AuthorityKeyIdentifier',[120,238,87,144,34,253,162,98|...], > asn1_NOVALUE,asn1_NOVALUE}}, > {'Extension',{2,5,29,19}, > false, > > {'BasicConstraints',true,asn1_NOVALUE}}]}, > {'SignatureAlgorithm',{1,2,840,113549,1,1,11},'NULL'}, > {0, > <<144,83,238,30,69,148,175,118,68,210,10,13,206,206,145, > 185,25,205,100,68,159,124,65,...>>}} > 4> > > > Dan. > > > On Thu, Jul 31, 2014 at 9:50 AM, Ingela Anderton Andin > > wrote: > > Hi! > > Sorry for the late answer. Was this pre 17.0 or 17.0 ? There was a > unicode fix in 17.0 the accidentally seems to have been lost in the > release notes. If you still have problems could you please send us a > sample cert that fails to speed up the process. > > Regards Ingela Erlang/OTP team - Ericsson AB > > > > > > On 06/12/2014 10:57 PM, Daniel Goertzen wrote: > > In the public_key application, decoding of attributes in x509 > certificates does not always decode to a string as indicated in the > documentation. The documentation says that the value of > commonName (and > several other attributes) should be: > > special_string() = {teletexString, string()} | {printableString, > string()} | {universalString, string()} | {utf8String, string()} | > {bmpString, string()} > > ... however when I decode a cert I see a utf8String coming out as a > binary instead of a string().... > > [{'AttributeTypeAndValue', > {2,5,4,3}, > {utf8String,<<"Daniel Goertzen">>}}], > > ... and typer shows several other non-string representations > (unicode > characters represented by 4-tuples): > > -spec dec_X520CommonName(_) -> > {'bmpString',[byte() | {byte(),byte(),byte(),byte()}] | > {byte(),binary()}} | > {'printableString',[byte() | {byte(),byte(),byte(),byte()}] | > {byte(),binary()}} | > {'teletexString',[byte() | {byte(),byte(),byte(),byte()}] | > {byte(),binary()}} | > {'universalString',[byte() | {byte(),byte(),byte(),byte()}] | > {byte(),binary()}} | > {'utf8String',_}. > > > > Also, encoding does not accept unicode strings (list of chars). The > example below crashes. > > Subject = {rdnSequence, [ > [#'AttributeTypeAndValue'{ > type = ?'id-at-commonName', > value = {utf8String, [16#4e09|" string starting with a > chinese symbol"]} > }] > ]}, > > > > I assume that the documentation is right and the decoded > representation > should be "list of characters". Instead internal > representations are > coming through. > > > _________________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/__listinfo/erlang-bugs > > > > _________________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/__listinfo/erlang-bugs > > > > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > From magnus.ottenklinger@REDACTED Fri Sep 5 09:10:36 2014 From: magnus.ottenklinger@REDACTED (Magnus Ottenklinger) Date: Fri, 5 Sep 2014 07:10:36 +0000 Subject: [erlang-bugs] Deadlocking application_controller using init:stop/1, 2 Message-ID: Hey List, I sent the following mail yesterday, but it seems that the mail was lost in space. Apologies if this reaches you twice. Magnus ----- We encountered a deadlock-problem with Erlang's application mechanism in at least R16B01 and OTP 17. Namely, when using init:stop/0,1, a certain order of events can deadlock the application_controller, leaving the VM running. This can be mitigated using the Kernel application's shutdown_timeout, but we think that this would just be a workaround and not a good fix. Consider the following simple (and contrived) example: -module(simple_app). -behaviour(application). -export([start/2, stop/1]). start(_Type=normal, _Args) -> {ok,self()}. stop(_State) -> application:load(crypto), %% yes, this is a bad idea. ok. Compile and run the example: 1> make:all([load]). up_to_date 2> application:start(simple). ok 3> init:stop(). ok 4> After this, application_controller will wait for simple_app to shutdown, whereas simple_app waits for the application_controller. Calls like application:which_applications/0 will time out. Note that we encountered the problem above in a more complicated setting: while starting our system, we received init:stop/0,1 from user interaction with the shell. As some processes were still starting up (and calling application:load/1 in that process), we got stuck in a VM which could not stop itself using init:stop/0,1 anymore. I propose that application:load/1,2 (and probably application:unload/1,2 as well) should return {error,shutting_down} if the application_controller is currently shutting down. For this, the application_controller should be able to reply to application:load_application/1 within its blocking receive in application:terminate/2. Of course, as mentioned above, using shutdown_timeout would be possible, but that might crash applications unnecessarily when they would be able to shutdown nicely. Regards, Magnus -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Fri Sep 5 15:42:41 2014 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 5 Sep 2014 15:42:41 +0200 Subject: [erlang-bugs] Deadlocking application_controller using init:stop/1, 2 In-Reply-To: References: Message-ID: Hi Magnus! I've started looking at this, and I see that application_controller would need to handle much more than just load/unload requests in the blocking receive in order so solve this problem. There are quite a few calls in application_controller that have the timer set to infinity. I'm not saying that this can/shall not be done, but I think we need to dig a bit deeper into the problem before we decide what to do. If it shall be fixed in the code or if better documentation might be sufficient. You say that the problem was originally detected while starting the system and receiving an init:stop via user interaction. Would it be possible to provide some further details about this? Do you have applications that load/start/stop other applications? Regards /siri@REDACTED 2014-09-05 9:10 GMT+02:00 Magnus Ottenklinger < magnus.ottenklinger@REDACTED>: > Hey List, > > > > I sent the following mail yesterday, but it seems that the mail was lost > in space. Apologies if this reaches you twice. > > > > Magnus > > > > ----- > > > > We encountered a deadlock-problem with Erlang?s application mechanism in > at least R16B01 and OTP 17. Namely, when using init:stop/0,1, a certain > order of events can deadlock the application_controller, leaving the VM > running. This can be mitigated using the Kernel application?s > shutdown_timeout, but we think that this would just be a workaround and not > a good fix. > > > > Consider the following simple (and contrived) example: > > > > -module(simple_app). > > > > -behaviour(application). > > > > -export([start/2, > > stop/1]). > > > > start(_Type=normal, _Args) -> > > {ok,self()}. > > > > stop(_State) -> > > application:load(crypto), %% yes, this is a bad idea. > > ok. > > > > Compile and run the example: > > > > 1> make:all([load]). > > up_to_date > > 2> application:start(simple). > > ok > > 3> init:stop(). > > ok > > 4> > > > > After this, application_controller will wait for simple_app to shutdown, > whereas simple_app waits for the application_controller. Calls like > application:which_applications/0 will time out. > > > > Note that we encountered the problem above in a more complicated setting: > while starting our system, we received init:stop/0,1 from user interaction > with the shell. As some processes were still starting up (and calling > application:load/1 in that process), we got stuck in a VM which could not > stop itself using init:stop/0,1 anymore. > > > > I propose that application:load/1,2 (and probably application:unload/1,2 > as well) should return {error,shutting_down} if the application_controller > is currently shutting down. For this, the application_controller should be > able to reply to application:load_application/1 within its blocking receive > in application:terminate/2. > > > > Of course, as mentioned above, using shutdown_timeout would be possible, > but that might crash applications unnecessarily when they would be able to > shutdown nicely. > > > > > > Regards, Magnus > > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.ottenklinger@REDACTED Tue Sep 9 11:38:06 2014 From: magnus.ottenklinger@REDACTED (Magnus Ottenklinger) Date: Tue, 9 Sep 2014 09:38:06 +0000 Subject: [erlang-bugs] Deadlocking application_controller using init:stop/1, 2 Message-ID: <71607aa8a2fd48eb95938c1db5552c4a@DBXPR03MB511.eurprd03.prod.outlook.com> Hey Siri, sorry for taking so long to reply. Our system takes quite some time starting up (around one minute). While this is being done, multiple applications are started, each with a supervisor tree. Within those supervisor trees, processes might start other OTP applications, such as ssl. The init:stop() is sent to the VM by our /etc/init.d script. If e.g. an error is detected during the startup phase, and we want to stop the node, the described deadlock appears, rendering the system unstoppable (in a clean way). Regards, Magnus From erlangsiri@REDACTED Wed Sep 10 16:32:19 2014 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 10 Sep 2014 16:32:19 +0200 Subject: [erlang-bugs] Deadlocking application_controller using init:stop/1, 2 In-Reply-To: <71607aa8a2fd48eb95938c1db5552c4a@DBXPR03MB511.eurprd03.prod.outlook.com> References: <71607aa8a2fd48eb95938c1db5552c4a@DBXPR03MB511.eurprd03.prod.outlook.com> Message-ID: Thanks for the additional information, Magnus! We will discuss this a bit more in the team before proceeding. Regards /siri 2014-09-09 11:38 GMT+02:00 Magnus Ottenklinger < magnus.ottenklinger@REDACTED>: > Hey Siri, > > sorry for taking so long to reply. > > Our system takes quite some time starting up (around one minute). While > this is being done, multiple applications are started, each with a > supervisor tree. Within those supervisor trees, processes might start other > OTP applications, such as ssl. > > The init:stop() is sent to the VM by our /etc/init.d script. If e.g. an > error is detected during the startup phase, and we want to stop the node, > the described deadlock appears, rendering the system unstoppable (in a > clean way). > > Regards, Magnus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Sep 10 20:49:55 2014 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 10 Sep 2014 11:49:55 -0700 Subject: [erlang-bugs] CT test crash Message-ID: <54109D53.80802@gmail.com> During a CT test shutdown (all tests were successful) I had CT crash with HTML output pending. This happened with Erlang 17.0, so I am not sure if this has been fixed already. Example output is below: (...many of these ERROR REPORT terminated io format lines, full output at https://travis-ci.org/CloudI/CloudI/jobs/34936288...) =ERROR REPORT==== 10-Sep-2014::17:56:01 === Error in process <0.356.1> on node 'test@REDACTED' with exit value: {terminated,[{io,format,[<0.128.0>,"~ts",[[[[[10,60,100,105,118,32,99,108,97,115,115,61,34,101,114,114,111,114,95,108,111,103,103,101,114,34,62,60,98,62,42,42,42,32,83,121,115,116,101,109,32,114,101,112,111,114,116,32,100,117,114,105,110,103,32,"cloudi_service_quorum_SUITE",58,"end_per_suite",47,49,32,50,48,49,52,45,48,57,45,49,48,32,49,55,58,53,54,58,48,48,46,52,54,51,32,42,42,42,60,47,98,62]],"\n",["\n",61,"INFO REPORT",61,61,61,61,32,"10",45,"Sep",45,"... =ERROR REPORT==== 10-Sep-2014::17:56:01 === Error in process <0.357.1> on node 'test@REDACTED' with exit value: {terminated,[{io,format,[<0.128.0>,"~ts",[[[[[10,60,100,105,118,32,99,108,97,115,115,61,34,101,114,114,111,114,95,108,111,103,103,101,114,34,62,60,98,62,42,42,42,32,83,121,115,116,101,109,32,114,101,112,111,114,116,32,100,117,114,105,110,103,32,"cloudi_service_quorum_SUITE",58,"end_per_suite",47,49,32,50,48,49,52,45,48,57,45,49,48,32,49,55,58,53,54,58,48,48,46,52,54,52,32,42,42,42,60,47,98,62]],"\n",["\n",61,"INFO REPORT",61,61,61,61,32,"10",45,"Sep",45,"... =ERROR REPORT==== 10-Sep-2014::17:56:01 === Error in process <0.38.0> on node 'test@REDACTED' with exit value: {badarg,[{erlang,binary_to_term,[{error,does_not_exist}],[]},{ct_logs,get_cache_data,1,[{file,"ct_logs.erl"},{line,2607}]},{ct_logs,make_all_suites_index,1,[{file,"ct_logs.erl"},{line,2132}]},{ct_logs,close,2,[{file,"ct_logs.erl"},... Test run crashed! This could be an internal error - please report! {badarg,[{erlang,binary_to_term,[{error,does_not_exist}],[]}, {ct_logs,get_cache_data,1,[{file,"ct_logs.erl"},{line,2607}]}, {ct_logs,make_all_suites_index,1,[{file,"ct_logs.erl"},{line,2132}]}, {ct_logs,close,2,[{file,"ct_logs.erl"},{line,170}]}, {ct_util,loop,3,[{file,"ct_util.erl"},{line,450}]}]} From shayan@REDACTED Thu Sep 11 06:12:53 2014 From: shayan@REDACTED (Shayan Pooya) Date: Thu, 11 Sep 2014 00:12:53 -0400 Subject: [erlang-bugs] Dialyzer's incorrect warning about opaque types Message-ID: I have these two modules: -module(has_opaque). -export([init/0]). -opaque state() :: queue:queue(erlang:timestamp()). -export_type([state/0]). -spec init()-> state(). init() -> queue:new(). and: -module(use_opaque). -export([init/0]). -record(state, {t :: has_opaque:state()}). init() -> {ok, #state{t = has_opaque:init()}}. Dialyzer is giving me this error message: """ use_opaque.erl:7: The attempt to match a term of type 'undefined' | queue:queue({non_neg_integer(),non_neg_integer(),non_neg_integer()}) against the variable _ breaks the opaqueness of has_opaque:state() """ Compilation and invoking dialyzer: compile: mkdir -p ebin erlc +debug_info -o ebin/ *.erl dialyzer: dialyzer --get_warnings -Wno_return -Wunmatched_returns -Werror_handling -r ebin/ --plt ~/.dialyzer_plt I get this error happens in Erlang 17.1 and 17.2.2 (on Linux). But Erlang R15B02 on OpenBSD did not complain. -------------- next part -------------- An HTML attachment was scrubbed... URL: From peppe@REDACTED Thu Sep 11 10:25:24 2014 From: peppe@REDACTED (Peter Andersson) Date: Thu, 11 Sep 2014 10:25:24 +0200 Subject: [erlang-bugs] CT test crash In-Reply-To: <54109D53.80802@gmail.com> References: <54109D53.80802@gmail.com> Message-ID: <54115C74.9030808@erlang.org> Hi Michael, Thanks a lot for the report! We'll have a look at it ASAP. Best, Peter Ericsson AB, Erlang/OTP Michael Truog wrote: > During a CT test shutdown (all tests were successful) I had CT crash > with HTML output pending. This happened with Erlang 17.0, so I am not > sure if this has been fixed already. > > Example output is below: > > (...many of these ERROR REPORT terminated io format lines, full output > at https://travis-ci.org/CloudI/CloudI/jobs/34936288...) > > =ERROR REPORT==== 10-Sep-2014::17:56:01 === > > Error in process <0.356.1> on node 'test@REDACTED' with exit value: > {terminated,[{io,format,[<0.128.0>,"~ts",[[[[[10,60,100,105,118,32,99,108,97,115,115,61,34,101,114,114,111,114,95,108,111,103,103,101,114,34,62,60,98,62,42,42,42,32,83,121,115,116,101,109,32,114,101,112,111,114,116,32,100,117,114,105,110,103,32,"cloudi_service_quorum_SUITE",58,"end_per_suite",47,49,32,50,48,49,52,45,48,57,45,49,48,32,49,55,58,53,54,58,48,48,46,52,54,51,32,42,42,42,60,47,98,62]],"\n",["\n",61,"INFO > REPORT",61,61,61,61,32,"10",45,"Sep",45,"... > > =ERROR REPORT==== 10-Sep-2014::17:56:01 === > > Error in process <0.357.1> on node 'test@REDACTED' with exit value: > {terminated,[{io,format,[<0.128.0>,"~ts",[[[[[10,60,100,105,118,32,99,108,97,115,115,61,34,101,114,114,111,114,95,108,111,103,103,101,114,34,62,60,98,62,42,42,42,32,83,121,115,116,101,109,32,114,101,112,111,114,116,32,100,117,114,105,110,103,32,"cloudi_service_quorum_SUITE",58,"end_per_suite",47,49,32,50,48,49,52,45,48,57,45,49,48,32,49,55,58,53,54,58,48,48,46,52,54,52,32,42,42,42,60,47,98,62]],"\n",["\n",61,"INFO > REPORT",61,61,61,61,32,"10",45,"Sep",45,"... > > =ERROR REPORT==== 10-Sep-2014::17:56:01 === > > Error in process <0.38.0> on node 'test@REDACTED' with exit value: > {badarg,[{erlang,binary_to_term,[{error,does_not_exist}],[]},{ct_logs,get_cache_data,1,[{file,"ct_logs.erl"},{line,2607}]},{ct_logs,make_all_suites_index,1,[{file,"ct_logs.erl"},{line,2132}]},{ct_logs,close,2,[{file,"ct_logs.erl"},... > > > Test run crashed! This could be an internal error - please report! > > {badarg,[{erlang,binary_to_term,[{error,does_not_exist}],[]}, > > {ct_logs,get_cache_data,1,[{file,"ct_logs.erl"},{line,2607}]}, > > {ct_logs,make_all_suites_index,1,[{file,"ct_logs.erl"},{line,2132}]}, > > {ct_logs,close,2,[{file,"ct_logs.erl"},{line,170}]}, > > {ct_util,loop,3,[{file,"ct_util.erl"},{line,450}]}]} > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > > From alex@REDACTED Fri Sep 12 14:23:02 2014 From: alex@REDACTED (Alex Wilson) Date: Fri, 12 Sep 2014 22:23:02 +1000 Subject: [erlang-bugs] Segfault in asn1rt_nif:decode_ber_tlv Message-ID: <5412E5A6.7020202@cooperi.net> Hi erlang-bugs, I've found a very interesting segmentation fault bug in asn1rt_nif:decode_ber_tlv. A simple case to reproduce it: just type this at the shell: asn1rt_nif:decode_ber_tlv(<<"!",16#80>>). It seems like whether or not the segmentation fault manifests varies a lot between different OS, compiler and OTP versions. So far I've reproduced it on: * R16B03-1 on Mac OSX Mavericks (clang) * R16B01 on OpenBSD 5.3-stable with gcc 4.2.1 * R16B03-1 on Linux (Fedora) with gcc 4.8.3 * R17B01 on OpenBSD 5.6-current with gcc 4.2.1 * R17 developer build from "maint" branch as of last week, Mac OSX Mavericks (clang) I also asked some random people in #erlang on freenode to try it out and they also reproduced the segfault using my test case. Sometimes it doesn't segfault the first time around, but if you run it a few times at the shell it will do it eventually. Backtrace from R17 maint on Mac OSX: * thread #1: tid = 0x2410f5, 0x00007fff958db1aa libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x107ffff0) * frame #0: 0x00007fff958db1aa libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458 frame #1: 0x000000001037aa49 asn1rt_nif.so`decode_ber_tlv_raw [inlined] ber_decode_begin(env=0x00007fff5fbff548, in_buf=, in_buf_len=, err_pos=) + 80 at asn1_erl_nif.c:854 frame #2: 0x000000001037a9f9 asn1rt_nif.so`decode_ber_tlv_raw(env=0x00007fff5fbff548, argc=, argv=) + 41 at asn1_erl_nif.c:1256 frame #3: 0x00000000100e4cbd beam`process_main + 58077 at beam_emu.c:3524 frame #4: 0x00000000100196dd beam`erl_start(argc=21, argv=) + 5997 at erl_init.c:1990 frame #5: 0x0000000010000df9 beam`main(argc=, argv=) + 9 at erl_main.c:29 Backtrace from R16B03-1 on Linux (gdb): (no symbols on that machine, sorry, but can clearly see it's the same trace) #0 __memcpy_sse2_unaligned () at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:157 #1 0x00007f0c48cbdace in ber_decode_begin () from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so #2 0x00007f0c48cbdb6f in decode_ber_tlv_raw () from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so #3 0x0000000000533254 in process_main () #4 0x00000000004508a0 in erl_start () #5 0x0000000000434049 in main () There's a lot of inlining and optimisation going on in this part of the code, which makes it hard to look back and forth between the assembly and C. Anyway, the segfault is caused by running off the end of memory after doing a memcpy with -2 as the length (0xfffffffe). This is because ib_index has gotten to a value of 4 when the in_buf_len is only 2. Using some watchpoints I figured out that ber_decode_value's if (indef == 1) block is responsible for the incrementing of ib_index beyond the end of the binary. I hacked up the following patch: --- a/lib/asn1/c_src/asn1_erl_nif.c +++ b/lib/asn1/c_src/asn1_erl_nif.c @@ -968,16 +968,16 @@ static int ber_decode_value(ErlNifEnv* env, ERL_NIF_TERM *value, unsigned cha if (indef == 1) { /* in this case it is desireably to check that indefinite length end bytes exist in inbuffer */ curr_head = enif_make_list(env, 0); - while (!(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { - if (*ib_index >= in_buf_len) - return ASN1_INDEF_LEN_ERROR; - + while ((*ib_index + 1 < in_buf_len) && + !(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { if ((maybe_ret = ber_decode(env, &term, in_buf, ib_index, in_buf_len)) <= ASN1_ERROR ) return maybe_ret; curr_head = enif_make_list_cell(env, term, curr_head); } + if (*ib_index + 1 >= in_buf_len) + return ASN1_INDEF_LEN_ERROR; enif_make_reverse_list(env, curr_head, value); (*ib_index) += 2; /* skip the indefinite length end bytes */ } else if (form == ASN1_CONSTRUCTED) And it seems to stop it happening with my test case on OSX, at least. It makes two changes -- checking for *ib_index + 1 >= in_buf_len (because it's using in_buf[*ib_index + 1], it should check that _that_ index is valid, not just *ib_index). The second change is to move the check from an "if" inside the loop to a loop condition. The movement to the loop condition seems to be the main difference. Looking at the assembly, it seems like the compiler is reasoning (after inlining the entire ber_decode function into this loop) that it can hoist that if out somehow?... maybe because the ber_decode code will already check it (???) Putting it into the loop condition seems to stop this behaviour. But if I've misdiagnosed the problem (quite possible), then perhaps all I've done is permute the code just enough to stop the optimiser doing this one thing for now, and there are other problems just waiting in the wings... Hoping somebody who knows the asn1 code better than I do (or C in general) might be able to help! From hans.bolinder@REDACTED Tue Sep 16 09:35:00 2014 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Tue, 16 Sep 2014 07:35:00 +0000 Subject: [erlang-bugs] Dialyzer's incorrect warning about opaque types In-Reply-To: References: Message-ID: <56466BD70414EA48969B4064696CF28C21DEF0BC@ESESSMB207.ericsson.se> Hi, It's possible that Dialyzer will be able to handle "towers" of opaque types better in the future, but don't hold your breath. It's (probably) quite tricky to implement, and has currently no priority. For now I'll just repeat the piece of advice that Kostis usually puts forward when questions like this arise: tag your type. Like: -opaque state() :: {some_tag, queue:queue(erlang:timestamp())}. Best regards, Hans Bolinder, Erlang OTP/team, Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Tue Sep 16 21:26:18 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 16 Sep 2014 21:26:18 +0200 Subject: [erlang-bugs] enif_make_int64 => -0 for INT64_MIN with gcc 4.9.1 In-Reply-To: <53EDD70C.4010406@erlang.org> References: <53DF5BC7.7010002@erlang.org> <5263DD26-BFF3-429C-957A-24BE27669394@gmail.com> <53EDC492.6050809@erlang.org> <21485.51186.785472.310413@gargle.gargle.HOWL> <53EDD70C.4010406@erlang.org> Message-ID: On Fri, Aug 15, 2014 at 11:46 AM, Lukas Larsson wrote: > Thanks for pointing this out! It would appear that when I hit UB, > deamons might fly out of my nose. I'll fix the other UBs I've found > as well. Lukas, are all fixes in maint? Whenever I use 17.2.2-UBSan-amd64, it prints -0. From tony@REDACTED Wed Sep 17 00:18:28 2014 From: tony@REDACTED (Tony Rogvall) Date: Wed, 17 Sep 2014 00:18:28 +0200 Subject: [erlang-bugs] try in shell Message-ID: <3A5A53EA-FCDF-4309-BA22-D831076F533F@rogvall.se> Hi! Not sure how this is supposed to be, given that there is a shell function called catch_exception/1. Running in 17.1 (approx) > try throw(x) catch Class:Reason -> {Class,Reason} end. {throw,x} > try exit(2) catch Class:Reason -> {Class,Reason} end. * exception exit: 2 > try foo:bar() catch Class:Reason -> {Class,Reason} end. * exception error: undefined function erl_eval:expr/3 > try 1/0 catch Class:Reason -> {Class,Reason} end. * exception error: an error occurred when evaluating an arithmetic expression > try (a=b) catch Class:Reason -> {Class,Reason} end. * exception error: no match of right hand side value b Putting the above try expression in a module: -module(tryme). -compile(export_all). test1() -> try throw(x) catch Class:Reason -> {Class,Reason} end. test2() -> try exit(2) catch Class:Reason -> {Class,Reason} end. test3() -> try (a=b) catch Class:Reason -> {Class,Reason} end. test4() -> try 1/0 catch Class:Reason -> {Class,Reason} end. test5() -> try foo:bar() catch Class:Reason -> {Class,Reason} end. First compile: tryme.erl:11: Warning: no clause will ever match tryme.erl:14: Warning: this expression will fail with a 'badarith' exception On line 11 there are no try clauses, but catch clauses that will always match, so the warning is a bit strange. And the expression on line 14 is a try, so it will never fail with badarith, possibly return an error tuple lets see: > tryme:test1(). {throw,x} > tryme:test2(). {exit,2} > tryme:test3(). {error,{badmatch,b}} > tryme:test4(). {error,badarith} > tryme:test5(). {error,undef} This looks a bit more like I would expect. Comments? /Tony From daniel@REDACTED Wed Sep 17 00:46:54 2014 From: daniel@REDACTED (Daniel Luna) Date: Tue, 16 Sep 2014 18:46:54 -0400 Subject: [erlang-bugs] try in shell In-Reply-To: <3A5A53EA-FCDF-4309-BA22-D831076F533F@rogvall.se> References: <3A5A53EA-FCDF-4309-BA22-D831076F533F@rogvall.se> Message-ID: The answer to your question is called f(). Your variables are bound in all future catch statements and won't match. The errors from compilation is about the *statement*, not the full function. The statement will result in an arithmetic error even if the function won't. Your third example in the shell leading to the erl_eval:expr/3 result looks like a bug in the shell though. I'm guessing that the shell strips one layer of stack trace to show the original function, but this solution leads to the wrong result when there's an extra level of indirection. Regards, Daniel On Sep 16, 2014 6:18 PM, "Tony Rogvall" wrote: > Hi! > > Not sure how this is supposed to be, given that there is a shell function > called catch_exception/1. > > Running in 17.1 (approx) > > > try throw(x) catch Class:Reason -> {Class,Reason} end. > {throw,x} > > > try exit(2) catch Class:Reason -> {Class,Reason} end. > * exception exit: 2 > > > try foo:bar() catch Class:Reason -> {Class,Reason} end. > * exception error: undefined function erl_eval:expr/3 > > > try 1/0 catch Class:Reason -> {Class,Reason} end. > * exception error: an error occurred when evaluating an arithmetic > expression > > > try (a=b) catch Class:Reason -> {Class,Reason} end. > * exception error: no match of right hand side value b > > Putting the above try expression in a module: > > -module(tryme). > -compile(export_all). > > test1() -> > try throw(x) catch Class:Reason -> {Class,Reason} end. > > test2() -> > try exit(2) catch Class:Reason -> {Class,Reason} end. > > test3() -> > try (a=b) catch Class:Reason -> {Class,Reason} end. > > test4() -> > try 1/0 catch Class:Reason -> {Class,Reason} end. > > test5() -> > try foo:bar() catch Class:Reason -> {Class,Reason} end. > > > First compile: > tryme.erl:11: Warning: no clause will ever match > tryme.erl:14: Warning: this expression will fail with a 'badarith' > exception > > On line 11 there are no try clauses, but catch clauses that will always > match, so the warning is a bit strange. > And the expression on line 14 is a try, so it will never fail with > badarith, possibly return an error tuple lets see: > > > tryme:test1(). > {throw,x} > > tryme:test2(). > {exit,2} > > tryme:test3(). > {error,{badmatch,b}} > > tryme:test4(). > {error,badarith} > > tryme:test5(). > {error,undef} > > This looks a bit more like I would expect. > > Comments? > > /Tony > > > > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shayan@REDACTED Wed Sep 17 02:33:33 2014 From: shayan@REDACTED (Shayan Pooya) Date: Tue, 16 Sep 2014 20:33:33 -0400 Subject: [erlang-bugs] Dialyzer's incorrect warning about opaque types In-Reply-To: <56466BD70414EA48969B4064696CF28C21DEF0BC@ESESSMB207.ericsson.se> References: <56466BD70414EA48969B4064696CF28C21DEF0BC@ESESSMB207.ericsson.se> Message-ID: Thanks for the note Hans. I'll tag the type. However, the older versions of Erlang (before 17.0) did not complain about this piece of code. On Tue, Sep 16, 2014 at 3:35 AM, Hans Bolinder wrote: > Hi, > > It's possible that Dialyzer will be able to handle "towers" of opaque > types better in the future, but don't hold your breath. It's > (probably) quite tricky to implement, and has currently no priority. > > For now I'll just repeat the piece of advice that Kostis usually puts > forward when questions like this arise: tag your type. Like: > > -opaque state() :: {some_tag, queue:queue(erlang:timestamp())}. > > Best regards, > > Hans Bolinder, Erlang OTP/team, Ericsson > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Wed Sep 17 09:18:50 2014 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 17 Sep 2014 09:18:50 +0200 Subject: [erlang-bugs] enif_make_int64 => -0 for INT64_MIN with gcc 4.9.1 In-Reply-To: References: <53DF5BC7.7010002@erlang.org> <5263DD26-BFF3-429C-957A-24BE27669394@gmail.com> <53EDC492.6050809@erlang.org> <21485.51186.785472.310413@gargle.gargle.HOWL> <53EDD70C.4010406@erlang.org> Message-ID: <541935DA.6090908@erlang.org> Hello, It is merged in maint to be released in 17.3. Lukas On 16/09/14 21:26, Tuncer Ayaz wrote: > On Fri, Aug 15, 2014 at 11:46 AM, Lukas Larsson wrote: > >> Thanks for pointing this out! It would appear that when I hit UB, >> deamons might fly out of my nose. I'll fix the other UBs I've found >> as well. > Lukas, are all fixes in maint? > > Whenever I use 17.2.2-UBSan-amd64, it prints -0. > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > From tuncer.ayaz@REDACTED Wed Sep 17 12:10:26 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 17 Sep 2014 12:10:26 +0200 Subject: [erlang-bugs] enif_make_int64 => -0 for INT64_MIN with gcc 4.9.1 In-Reply-To: <541935DA.6090908@erlang.org> References: <53DF5BC7.7010002@erlang.org> <5263DD26-BFF3-429C-957A-24BE27669394@gmail.com> <53EDC492.6050809@erlang.org> <21485.51186.785472.310413@gargle.gargle.HOWL> <53EDD70C.4010406@erlang.org> <541935DA.6090908@erlang.org> Message-ID: On Wed, Sep 17, 2014 at 9:18 AM, Lukas Larsson wrote: > Hello, > > It is merged in maint to be released in 17.3. Thanks, 17.2.2 wasn't new enough: FAIL OTP-17.2.2 GOOD OTP-17.2.2-235-g6e2fd45 (maint) GOOD OTP-17.2.2-587-g08fb380 (master) From tony@REDACTED Wed Sep 17 13:50:44 2014 From: tony@REDACTED (Tony Rogvall) Date: Wed, 17 Sep 2014 13:50:44 +0200 Subject: [erlang-bugs] try in shell In-Reply-To: References: <3A5A53EA-FCDF-4309-BA22-D831076F533F@rogvall.se> Message-ID: <8465DB48-EA6D-40B3-B91D-AA3A088ABC4C@rogvall.se> On 17 sep 2014, at 00:46, Daniel Luna wrote: > The answer to your question is called f(). > > You are right, I am a fool :-) > Your variables are bound in all future catch statements and won't match. > > The errors from compilation is about the *statement*, not the full function. The statement will result in an arithmetic error even if the function won't. > Well. I think that is up for an argument. The compiler is "smart" enough to see that the code will result in an arithmetic exception but not even smart enough to see that it is caught? > Your third example in the shell leading to the erl_eval:expr/3 result looks like a bug in the shell though. I'm guessing that the shell strips one layer of stack trace to show the original function, but this solution leads to the wrong result when there's an extra level of indirection. > > Hmm. /Tony > Regards, > > Daniel > > On Sep 16, 2014 6:18 PM, "Tony Rogvall" wrote: > Hi! > > Not sure how this is supposed to be, given that there is a shell function called catch_exception/1. > > Running in 17.1 (approx) > > > try throw(x) catch Class:Reason -> {Class,Reason} end. > {throw,x} > > > try exit(2) catch Class:Reason -> {Class,Reason} end. > * exception exit: 2 > > > try foo:bar() catch Class:Reason -> {Class,Reason} end. > * exception error: undefined function erl_eval:expr/3 > > > try 1/0 catch Class:Reason -> {Class,Reason} end. > * exception error: an error occurred when evaluating an arithmetic expression > > > try (a=b) catch Class:Reason -> {Class,Reason} end. > * exception error: no match of right hand side value b > > Putting the above try expression in a module: > > -module(tryme). > -compile(export_all). > > test1() -> > try throw(x) catch Class:Reason -> {Class,Reason} end. > > test2() -> > try exit(2) catch Class:Reason -> {Class,Reason} end. > > test3() -> > try (a=b) catch Class:Reason -> {Class,Reason} end. > > test4() -> > try 1/0 catch Class:Reason -> {Class,Reason} end. > > test5() -> > try foo:bar() catch Class:Reason -> {Class,Reason} end. > > > First compile: > tryme.erl:11: Warning: no clause will ever match > tryme.erl:14: Warning: this expression will fail with a 'badarith' exception > > On line 11 there are no try clauses, but catch clauses that will always match, so the warning is a bit strange. > And the expression on line 14 is a try, so it will never fail with badarith, possibly return an error tuple lets see: > > > tryme:test1(). > {throw,x} > > tryme:test2(). > {exit,2} > > tryme:test3(). > {error,{badmatch,b}} > > tryme:test4(). > {error,badarith} > > tryme:test5(). > {error,undef} > > This looks a bit more like I would expect. > > Comments? > > /Tony > > > > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed Sep 17 14:17:04 2014 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 17 Sep 2014 14:17:04 +0200 Subject: [erlang-bugs] try in shell In-Reply-To: <8465DB48-EA6D-40B3-B91D-AA3A088ABC4C@rogvall.se> References: <3A5A53EA-FCDF-4309-BA22-D831076F533F@rogvall.se> <8465DB48-EA6D-40B3-B91D-AA3A088ABC4C@rogvall.se> Message-ID: On Wed, Sep 17, 2014 at 1:50 PM, Tony Rogvall wrote: > > On 17 sep 2014, at 00:46, Daniel Luna wrote: > > The errors from compilation is about the *statement*, not the full > function. The statement will result in an arithmetic error even if the > function won't. > > Well. I think that is up for an argument. The compiler is "smart" enough > to see that the code will result in an arithmetic exception > but not even smart enough to see that it is caught? > I like to call them subexpressions, not statements (this ain't C), but yes. It's easy to check every subexpression to see if it can be evaluated statically, and in that case, if it would definitely fail. To see that it would be caught requires also tracking the execution context for each subexpression and not reporting the failing code, not all that difficult but slightly messy. /Richard > > Regards, > > Daniel > On Sep 16, 2014 6:18 PM, "Tony Rogvall" wrote: > >> Hi! >> >> Not sure how this is supposed to be, given that there is a shell function >> called catch_exception/1. >> >> Running in 17.1 (approx) >> >> > try throw(x) catch Class:Reason -> {Class,Reason} end. >> {throw,x} >> >> > try exit(2) catch Class:Reason -> {Class,Reason} end. >> * exception exit: 2 >> >> > try foo:bar() catch Class:Reason -> {Class,Reason} end. >> * exception error: undefined function erl_eval:expr/3 >> >> > try 1/0 catch Class:Reason -> {Class,Reason} end. >> * exception error: an error occurred when evaluating an arithmetic >> expression >> >> > try (a=b) catch Class:Reason -> {Class,Reason} end. >> * exception error: no match of right hand side value b >> >> Putting the above try expression in a module: >> >> -module(tryme). >> -compile(export_all). >> >> test1() -> >> try throw(x) catch Class:Reason -> {Class,Reason} end. >> >> test2() -> >> try exit(2) catch Class:Reason -> {Class,Reason} end. >> >> test3() -> >> try (a=b) catch Class:Reason -> {Class,Reason} end. >> >> test4() -> >> try 1/0 catch Class:Reason -> {Class,Reason} end. >> >> test5() -> >> try foo:bar() catch Class:Reason -> {Class,Reason} end. >> >> >> First compile: >> tryme.erl:11: Warning: no clause will ever match >> tryme.erl:14: Warning: this expression will fail with a 'badarith' >> exception >> >> On line 11 there are no try clauses, but catch clauses that will always >> match, so the warning is a bit strange. >> And the expression on line 14 is a try, so it will never fail with >> badarith, possibly return an error tuple lets see: >> >> > tryme:test1(). >> {throw,x} >> > tryme:test2(). >> {exit,2} >> > tryme:test3(). >> {error,{badmatch,b}} >> > tryme:test4(). >> {error,badarith} >> > tryme:test5(). >> {error,undef} >> >> This looks a bit more like I would expect. >> >> Comments? >> >> /Tony >> >> >> >> >> >> _______________________________________________ >> erlang-bugs mailing list >> erlang-bugs@REDACTED >> http://erlang.org/mailman/listinfo/erlang-bugs >> > > "Installing applications can lead to corruption over time. Applications > gradually write over each other's libraries, partial upgrades occur, user > and system errors happen, and minute changes may be unnoticeable and > difficult to fix" > > > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Wed Sep 17 14:33:55 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 17 Sep 2014 14:33:55 +0200 Subject: [erlang-bugs] enif_make_int64 => -0 for INT64_MIN with gcc 4.9.1 In-Reply-To: References: <53DF5BC7.7010002@erlang.org> <5263DD26-BFF3-429C-957A-24BE27669394@gmail.com> <53EDC492.6050809@erlang.org> <21485.51186.785472.310413@gargle.gargle.HOWL> <53EDD70C.4010406@erlang.org> <541935DA.6090908@erlang.org> Message-ID: On Wed, Sep 17, 2014 at 12:10 PM, Tuncer Ayaz wrote: > On Wed, Sep 17, 2014 at 9:18 AM, Lukas Larsson wrote: > > Hello, > > > > It is merged in maint to be released in 17.3. > > Thanks, 17.2.2 wasn't new enough: > > FAIL OTP-17.2.2 > GOOD OTP-17.2.2-235-g6e2fd45 (maint) > GOOD OTP-17.2.2-587-g08fb380 (master) Building OTP-17.3 with --enable-sanitizers=undefined, you can see the following during the build of lib/wx/examples/demo: ERLC ex_aui.beam beam/bif.c:2828:5: runtime error: signed integer overflow: 426141286 * 10 cannot be represented in type 'int' I thought this was also fixed. From tuncer.ayaz@REDACTED Wed Sep 17 19:23:26 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 17 Sep 2014 19:23:26 +0200 Subject: [erlang-bugs] UBsan int overflows Message-ID: On Wed, Sep 17, 2014 at 2:33 PM, Tuncer Ayaz wrote: > Building OTP-17.3 with --enable-sanitizers=undefined, you can see > the following during the build of lib/wx/examples/demo: > ERLC ex_aui.beam > beam/bif.c:2828:5: runtime error: signed integer overflow: > 426141286 * 10 cannot be represented in type 'int' > > I thought this was also fixed. More during dialyzer --build_plt: beam/erl_process.c:9024:2: runtime error: signed integer overflow: 2147482509 + 2001 cannot be represented in type 'int' beam/erl_process.c:4908:35: runtime error: signed integer overflow: 3984 * 625175 cannot be represented in type 'int' From lukas@REDACTED Mon Sep 22 12:02:29 2014 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 22 Sep 2014 12:02:29 +0200 Subject: [erlang-bugs] UBsan int overflows In-Reply-To: References: Message-ID: <541FF3B5.8000304@erlang.org> On 17/09/14 19:23, Tuncer Ayaz wrote: > On Wed, Sep 17, 2014 at 2:33 PM, Tuncer Ayaz wrote: >> Building OTP-17.3 with --enable-sanitizers=undefined, you can see >> the following during the build of lib/wx/examples/demo: >> ERLC ex_aui.beam >> beam/bif.c:2828:5: runtime error: signed integer overflow: >> 426141286 * 10 cannot be represented in type 'int' >> >> I thought this was also fixed. As did I. I wonder where that fix went... I've created a new fix for it. > More during dialyzer --build_plt: > beam/erl_process.c:9024:2: runtime error: signed integer overflow: > 2147482509 + 2001 cannot be represented in type 'int' > beam/erl_process.c:4908:35: runtime error: signed integer overflow: > 3984 * 625175 cannot be represented in type 'int' > I've noticed these as well and a couple of other ones in erl_process.c. I've added them to our backlog to fix. Lukas From zl9d97p02@REDACTED Mon Sep 22 15:50:13 2014 From: zl9d97p02@REDACTED (Simon Cornish) Date: Mon, 22 Sep 2014 15:50:13 +0200 Subject: [erlang-bugs] Segfault in asn1rt_nif:decode_ber_tlv In-Reply-To: <5412E5A6.7020202@cooperi.net> References: <5412E5A6.7020202@cooperi.net> Message-ID: <20023-1411393813-601401@sneakemail.com> Hi Alex, Your analysis is correct regarding that the underlying fault is reading beyond the end of the buffer The attached patch solves your problem in the same (although stylistically different) way. It also corrects the fault that a primitive cannot be encoded using an indefinite length. This is both according to X.690 and common sense! If I get some breathing room later this week I'll put together a pull request for this with some test. Regards, Simon On 12 September 2014 14:23, Alex Wilson alex-at-cooperi.net |erlang-mail/gmail| wrote: > Hi erlang-bugs, > > I've found a very interesting segmentation fault bug in > asn1rt_nif:decode_ber_tlv. > > A simple case to reproduce it: just type this at the shell: > > asn1rt_nif:decode_ber_tlv(<<"!",16#80>>). > > It seems like whether or not the segmentation fault manifests varies a lot > between different OS, compiler and OTP versions. So far I've reproduced it > on: > > * R16B03-1 on Mac OSX Mavericks (clang) > * R16B01 on OpenBSD 5.3-stable with gcc 4.2.1 > * R16B03-1 on Linux (Fedora) with gcc 4.8.3 > * R17B01 on OpenBSD 5.6-current with gcc 4.2.1 > * R17 developer build from "maint" branch as of last week, Mac OSX > Mavericks (clang) > > I also asked some random people in #erlang on freenode to try it out and > they also reproduced the segfault using my test case. > > Sometimes it doesn't segfault the first time around, but if you run it a few > times at the shell it will do it eventually. > > > Backtrace from R17 maint on Mac OSX: > > * thread #1: tid = 0x2410f5, 0x00007fff958db1aa > libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458, queue = > 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, > address=0x107ffff0) > * frame #0: 0x00007fff958db1aa > libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458 > frame #1: 0x000000001037aa49 asn1rt_nif.so`decode_ber_tlv_raw [inlined] > ber_decode_begin(env=0x00007fff5fbff548, in_buf=, > in_buf_len=, err_pos=) + 80 at asn1_erl_nif.c:854 > frame #2: 0x000000001037a9f9 > asn1rt_nif.so`decode_ber_tlv_raw(env=0x00007fff5fbff548, argc=, > argv=) + 41 at asn1_erl_nif.c:1256 > frame #3: 0x00000000100e4cbd beam`process_main + 58077 at > beam_emu.c:3524 > frame #4: 0x00000000100196dd beam`erl_start(argc=21, argv=) > + 5997 at erl_init.c:1990 > frame #5: 0x0000000010000df9 beam`main(argc=, > argv=) + 9 at erl_main.c:29 > > Backtrace from R16B03-1 on Linux (gdb): (no symbols on that machine, sorry, > but can clearly see it's the same trace) > > #0 __memcpy_sse2_unaligned () > at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:157 > #1 0x00007f0c48cbdace in ber_decode_begin () > from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so > #2 0x00007f0c48cbdb6f in decode_ber_tlv_raw () > from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so > #3 0x0000000000533254 in process_main () > #4 0x00000000004508a0 in erl_start () > #5 0x0000000000434049 in main () > > > There's a lot of inlining and optimisation going on in this part of the > code, which makes it hard to look back and forth between the assembly and C. > > Anyway, the segfault is caused by running off the end of memory after doing > a memcpy with -2 as the length (0xfffffffe). This is because ib_index has > gotten to a value of 4 when the in_buf_len is only 2. > > Using some watchpoints I figured out that ber_decode_value's if (indef == 1) > block is responsible for the incrementing of ib_index beyond the end of the > binary. I hacked up the following patch: > > > --- a/lib/asn1/c_src/asn1_erl_nif.c > +++ b/lib/asn1/c_src/asn1_erl_nif.c > @@ -968,16 +968,16 @@ static int ber_decode_value(ErlNifEnv* env, > ERL_NIF_TERM *value, unsigned cha > if (indef == 1) { /* in this case it is desireably to check that > indefinite length > end bytes exist in inbuffer */ > curr_head = enif_make_list(env, 0); > - while (!(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { > - if (*ib_index >= in_buf_len) > - return ASN1_INDEF_LEN_ERROR; > - > + while ((*ib_index + 1 < in_buf_len) && > + !(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { > if ((maybe_ret = ber_decode(env, &term, in_buf, ib_index, > in_buf_len)) > <= ASN1_ERROR > ) > return maybe_ret; > curr_head = enif_make_list_cell(env, term, curr_head); > } > + if (*ib_index + 1 >= in_buf_len) > + return ASN1_INDEF_LEN_ERROR; > enif_make_reverse_list(env, curr_head, value); > (*ib_index) += 2; /* skip the indefinite length end bytes */ > } else if (form == ASN1_CONSTRUCTED) > > > And it seems to stop it happening with my test case on OSX, at least. It > makes two changes -- checking for *ib_index + 1 >= in_buf_len (because it's > using in_buf[*ib_index + 1], it should check that _that_ index is valid, not > just *ib_index). The second change is to move the check from an "if" inside > the loop to a loop condition. > > The movement to the loop condition seems to be the main difference. Looking > at the assembly, it seems like the compiler is reasoning (after inlining the > entire ber_decode function into this loop) that it can hoist that if out > somehow?... maybe because the ber_decode code will already check it (???) > > Putting it into the loop condition seems to stop this behaviour. But if I've > misdiagnosed the problem (quite possible), then perhaps all I've done is > permute the code just enough to stop the optimiser doing this one thing for > now, and there are other problems just waiting in the wings... > > Hoping somebody who knows the asn1 code better than I do (or C in general) > might be able to help! > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs From zl9d97p02@REDACTED Tue Sep 23 10:24:43 2014 From: zl9d97p02@REDACTED (Simon Cornish) Date: Tue, 23 Sep 2014 10:24:43 +0200 Subject: [erlang-bugs] Segfault in asn1rt_nif:decode_ber_tlv In-Reply-To: <20023-1411393813-601401@sneakemail.com> References: <5412E5A6.7020202@cooperi.net> <20023-1411393813-601401@sneakemail.com> Message-ID: <16671-1411460683-615681@sneakemail.com> Mailer ate my patch :-( On 22 September 2014 15:50, Simon Cornish zl9d97p02-at-sneakemail.com |erlang-mail/gmail| <8rbn1sir5t@REDACTED> wrote: > Hi Alex, > Your analysis is correct regarding that the underlying fault is > reading beyond the end of the buffer > The attached patch solves your problem in the same (although > stylistically different) way. > It also corrects the fault that a primitive cannot be encoded using an > indefinite length. This is both according to X.690 and common sense! > If I get some breathing room later this week I'll put together a pull > request for this with some test. > Regards, > Simon > > > On 12 September 2014 14:23, Alex Wilson alex-at-cooperi.net > |erlang-mail/gmail| wrote: >> Hi erlang-bugs, >> >> I've found a very interesting segmentation fault bug in >> asn1rt_nif:decode_ber_tlv. >> >> A simple case to reproduce it: just type this at the shell: >> >> asn1rt_nif:decode_ber_tlv(<<"!",16#80>>). >> >> It seems like whether or not the segmentation fault manifests varies a lot >> between different OS, compiler and OTP versions. So far I've reproduced it >> on: >> >> * R16B03-1 on Mac OSX Mavericks (clang) >> * R16B01 on OpenBSD 5.3-stable with gcc 4.2.1 >> * R16B03-1 on Linux (Fedora) with gcc 4.8.3 >> * R17B01 on OpenBSD 5.6-current with gcc 4.2.1 >> * R17 developer build from "maint" branch as of last week, Mac OSX >> Mavericks (clang) >> >> I also asked some random people in #erlang on freenode to try it out and >> they also reproduced the segfault using my test case. >> >> Sometimes it doesn't segfault the first time around, but if you run it a few >> times at the shell it will do it eventually. >> >> >> Backtrace from R17 maint on Mac OSX: >> >> * thread #1: tid = 0x2410f5, 0x00007fff958db1aa >> libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458, queue = >> 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, >> address=0x107ffff0) >> * frame #0: 0x00007fff958db1aa >> libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458 >> frame #1: 0x000000001037aa49 asn1rt_nif.so`decode_ber_tlv_raw [inlined] >> ber_decode_begin(env=0x00007fff5fbff548, in_buf=, >> in_buf_len=, err_pos=) + 80 at asn1_erl_nif.c:854 >> frame #2: 0x000000001037a9f9 >> asn1rt_nif.so`decode_ber_tlv_raw(env=0x00007fff5fbff548, argc=, >> argv=) + 41 at asn1_erl_nif.c:1256 >> frame #3: 0x00000000100e4cbd beam`process_main + 58077 at >> beam_emu.c:3524 >> frame #4: 0x00000000100196dd beam`erl_start(argc=21, argv=) >> + 5997 at erl_init.c:1990 >> frame #5: 0x0000000010000df9 beam`main(argc=, >> argv=) + 9 at erl_main.c:29 >> >> Backtrace from R16B03-1 on Linux (gdb): (no symbols on that machine, sorry, >> but can clearly see it's the same trace) >> >> #0 __memcpy_sse2_unaligned () >> at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:157 >> #1 0x00007f0c48cbdace in ber_decode_begin () >> from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so >> #2 0x00007f0c48cbdb6f in decode_ber_tlv_raw () >> from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so >> #3 0x0000000000533254 in process_main () >> #4 0x00000000004508a0 in erl_start () >> #5 0x0000000000434049 in main () >> >> >> There's a lot of inlining and optimisation going on in this part of the >> code, which makes it hard to look back and forth between the assembly and C. >> >> Anyway, the segfault is caused by running off the end of memory after doing >> a memcpy with -2 as the length (0xfffffffe). This is because ib_index has >> gotten to a value of 4 when the in_buf_len is only 2. >> >> Using some watchpoints I figured out that ber_decode_value's if (indef == 1) >> block is responsible for the incrementing of ib_index beyond the end of the >> binary. I hacked up the following patch: >> >> >> --- a/lib/asn1/c_src/asn1_erl_nif.c >> +++ b/lib/asn1/c_src/asn1_erl_nif.c >> @@ -968,16 +968,16 @@ static int ber_decode_value(ErlNifEnv* env, >> ERL_NIF_TERM *value, unsigned cha >> if (indef == 1) { /* in this case it is desireably to check that >> indefinite length >> end bytes exist in inbuffer */ >> curr_head = enif_make_list(env, 0); >> - while (!(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { >> - if (*ib_index >= in_buf_len) >> - return ASN1_INDEF_LEN_ERROR; >> - >> + while ((*ib_index + 1 < in_buf_len) && >> + !(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { >> if ((maybe_ret = ber_decode(env, &term, in_buf, ib_index, >> in_buf_len)) >> <= ASN1_ERROR >> ) >> return maybe_ret; >> curr_head = enif_make_list_cell(env, term, curr_head); >> } >> + if (*ib_index + 1 >= in_buf_len) >> + return ASN1_INDEF_LEN_ERROR; >> enif_make_reverse_list(env, curr_head, value); >> (*ib_index) += 2; /* skip the indefinite length end bytes */ >> } else if (form == ASN1_CONSTRUCTED) >> >> >> And it seems to stop it happening with my test case on OSX, at least. It >> makes two changes -- checking for *ib_index + 1 >= in_buf_len (because it's >> using in_buf[*ib_index + 1], it should check that _that_ index is valid, not >> just *ib_index). The second change is to move the check from an "if" inside >> the loop to a loop condition. >> >> The movement to the loop condition seems to be the main difference. Looking >> at the assembly, it seems like the compiler is reasoning (after inlining the >> entire ber_decode function into this loop) that it can hoist that if out >> somehow?... maybe because the ber_decode code will already check it (???) >> >> Putting it into the loop condition seems to stop this behaviour. But if I've >> misdiagnosed the problem (quite possible), then perhaps all I've done is >> permute the code just enough to stop the optimiser doing this one thing for >> now, and there are other problems just waiting in the wings... >> >> Hoping somebody who knows the asn1 code better than I do (or C in general) >> might be able to help! >> _______________________________________________ >> erlang-bugs mailing list >> erlang-bugs@REDACTED >> http://erlang.org/mailman/listinfo/erlang-bugs > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs -------------- next part -------------- A non-text attachment was scrubbed... Name: gasn.patch Type: application/octet-stream Size: 1123 bytes Desc: not available URL: From tuncer.ayaz@REDACTED Tue Sep 23 11:05:06 2014 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 23 Sep 2014 11:05:06 +0200 Subject: [erlang-bugs] UBsan int overflows In-Reply-To: <541FF3B5.8000304@erlang.org> References: <541FF3B5.8000304@erlang.org> Message-ID: On Mon, Sep 22, 2014 at 12:02 PM, Lukas Larsson wrote: > > On 17/09/14 19:23, Tuncer Ayaz wrote: > > > > On Wed, Sep 17, 2014 at 2:33 PM, Tuncer Ayaz wrote: > > > > > > Building OTP-17.3 with --enable-sanitizers=undefined, you can see > > > the following during the build of lib/wx/examples/demo: > > > ERLC ex_aui.beam > > > beam/bif.c:2828:5: runtime error: signed integer overflow: > > > 426141286 * 10 cannot be represented in type 'int' > > > > > > I thought this was also fixed. > > As did I. I wonder where that fix went... I've created a new fix for it. > > > More during dialyzer --build_plt: > > beam/erl_process.c:9024:2: runtime error: signed integer overflow: > > 2147482509 + 2001 cannot be represented in type 'int' > > beam/erl_process.c:4908:35: runtime error: signed integer overflow: > > 3984 * 625175 cannot be represented in type 'int' > > > I've noticed these as well and a couple of other ones in > erl_process.c. I've added them to our backlog to fix. Thanks, and please keep us posted. From Tobias.Schlager@REDACTED Tue Sep 23 14:49:42 2014 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Tue, 23 Sep 2014 12:49:42 +0000 Subject: [erlang-bugs] mnesia subscription Message-ID: <12F2115FD1CCEE4294943B2608A18FA301223765F7@MAIL01.win.lbaum.eu> Hi, I just found out the hard way, that mnesia:subscribe/1 might lie to you, when you trap exits and are linked to other processes. Although running, Mnesia sometimes returns '{error,{node_not_running,nonode@REDACTED}}' when trying to subscribe. Even worse subsequent calls to subscribe from the same process tell you that the subscription actually took place (returning '{error,{already_exists,Category}}'). This is due to a receive statement located in mnesia_subscr:call/1 that tries to unify the functions return value. Unfortunately, it doesn't match for a specific process id, so every 'EXIT' message in the callers message queue will be consumed and trigger the false value. For a minimal test case fire up an erlang shell and do the following: 1> application:start(mnesia). ok 2> process_flag(trap_exit, true). false 3> spawn_link(fun() -> ok end). <0.52.0> 4> mnesia:subscribe(system). {error,{node_not_running,nonode@REDACTED}} 5> mnesia:subscribe(system). {error,{already_exists,system}} 6> mnesia:unsubscribe(system). {ok,nonode@REDACTED} 7> mnesia:subscribe(system). {ok,nonode@REDACTED} Regards Tobias From bjorn@REDACTED Wed Sep 24 16:06:20 2014 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 24 Sep 2014 16:06:20 +0200 Subject: [erlang-bugs] Segfault in asn1rt_nif:decode_ber_tlv In-Reply-To: <5412E5A6.7020202@cooperi.net> References: <5412E5A6.7020202@cooperi.net> Message-ID: Hi Alex, Thanks for the bug report and sorry for the late answer. We have included a fix for the problem in 17.3 which was released recently: OTP-12145 Robustness when decoding incorrect BER messages has been improved. Here is the commit that fixes the bug: https://github.com/erlang/otp/commit/7f385ebd984ed2931daa761819816b3e9da7d63c /Bjorn On Fri, Sep 12, 2014 at 2:23 PM, Alex Wilson wrote: > Hi erlang-bugs, > > I've found a very interesting segmentation fault bug in > asn1rt_nif:decode_ber_tlv. > > A simple case to reproduce it: just type this at the shell: > > asn1rt_nif:decode_ber_tlv(<<"!",16#80>>). > > It seems like whether or not the segmentation fault manifests varies a lot > between different OS, compiler and OTP versions. So far I've reproduced it > on: > > * R16B03-1 on Mac OSX Mavericks (clang) > * R16B01 on OpenBSD 5.3-stable with gcc 4.2.1 > * R16B03-1 on Linux (Fedora) with gcc 4.8.3 > * R17B01 on OpenBSD 5.6-current with gcc 4.2.1 > * R17 developer build from "maint" branch as of last week, Mac OSX > Mavericks (clang) > > I also asked some random people in #erlang on freenode to try it out and > they also reproduced the segfault using my test case. > > Sometimes it doesn't segfault the first time around, but if you run it a few > times at the shell it will do it eventually. > > > Backtrace from R17 maint on Mac OSX: > > * thread #1: tid = 0x2410f5, 0x00007fff958db1aa > libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458, queue = > 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, > address=0x107ffff0) > * frame #0: 0x00007fff958db1aa > libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458 > frame #1: 0x000000001037aa49 asn1rt_nif.so`decode_ber_tlv_raw [inlined] > ber_decode_begin(env=0x00007fff5fbff548, in_buf=, > in_buf_len=, err_pos=) + 80 at asn1_erl_nif.c:854 > frame #2: 0x000000001037a9f9 > asn1rt_nif.so`decode_ber_tlv_raw(env=0x00007fff5fbff548, argc=, > argv=) + 41 at asn1_erl_nif.c:1256 > frame #3: 0x00000000100e4cbd beam`process_main + 58077 at > beam_emu.c:3524 > frame #4: 0x00000000100196dd beam`erl_start(argc=21, argv=) > + 5997 at erl_init.c:1990 > frame #5: 0x0000000010000df9 beam`main(argc=, > argv=) + 9 at erl_main.c:29 > > Backtrace from R16B03-1 on Linux (gdb): (no symbols on that machine, sorry, > but can clearly see it's the same trace) > > #0 __memcpy_sse2_unaligned () > at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:157 > #1 0x00007f0c48cbdace in ber_decode_begin () > from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so > #2 0x00007f0c48cbdb6f in decode_ber_tlv_raw () > from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so > #3 0x0000000000533254 in process_main () > #4 0x00000000004508a0 in erl_start () > #5 0x0000000000434049 in main () > > > There's a lot of inlining and optimisation going on in this part of the > code, which makes it hard to look back and forth between the assembly and C. > > Anyway, the segfault is caused by running off the end of memory after doing > a memcpy with -2 as the length (0xfffffffe). This is because ib_index has > gotten to a value of 4 when the in_buf_len is only 2. > > Using some watchpoints I figured out that ber_decode_value's if (indef == 1) > block is responsible for the incrementing of ib_index beyond the end of the > binary. I hacked up the following patch: > > > --- a/lib/asn1/c_src/asn1_erl_nif.c > +++ b/lib/asn1/c_src/asn1_erl_nif.c > @@ -968,16 +968,16 @@ static int ber_decode_value(ErlNifEnv* env, > ERL_NIF_TERM *value, unsigned cha > if (indef == 1) { /* in this case it is desireably to check that > indefinite length > end bytes exist in inbuffer */ > curr_head = enif_make_list(env, 0); > - while (!(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { > - if (*ib_index >= in_buf_len) > - return ASN1_INDEF_LEN_ERROR; > - > + while ((*ib_index + 1 < in_buf_len) && > + !(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { > if ((maybe_ret = ber_decode(env, &term, in_buf, ib_index, > in_buf_len)) > <= ASN1_ERROR > ) > return maybe_ret; > curr_head = enif_make_list_cell(env, term, curr_head); > } > + if (*ib_index + 1 >= in_buf_len) > + return ASN1_INDEF_LEN_ERROR; > enif_make_reverse_list(env, curr_head, value); > (*ib_index) += 2; /* skip the indefinite length end bytes */ > } else if (form == ASN1_CONSTRUCTED) > > > And it seems to stop it happening with my test case on OSX, at least. It > makes two changes -- checking for *ib_index + 1 >= in_buf_len (because it's > using in_buf[*ib_index + 1], it should check that _that_ index is valid, not > just *ib_index). The second change is to move the check from an "if" inside > the loop to a loop condition. > > The movement to the loop condition seems to be the main difference. Looking > at the assembly, it seems like the compiler is reasoning (after inlining the > entire ber_decode function into this loop) that it can hoist that if out > somehow?... maybe because the ber_decode code will already check it (???) > > Putting it into the loop condition seems to stop this behaviour. But if I've > misdiagnosed the problem (quite possible), then perhaps all I've done is > permute the code just enough to stop the optimiser doing this one thing for > now, and there are other problems just waiting in the wings... > > Hoping somebody who knows the asn1 code better than I do (or C in general) > might be able to help! > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Wed Sep 24 16:11:29 2014 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 24 Sep 2014 16:11:29 +0200 Subject: [erlang-bugs] Segfault in asn1rt_nif:decode_ber_tlv In-Reply-To: <20023-1411393813-601401@sneakemail.com> References: <5412E5A6.7020202@cooperi.net> <20023-1411393813-601401@sneakemail.com> Message-ID: Hi Simon, As you can see in my late answer to Alex, we have fixed the bug. But I did not add an test for a primitive type encoded with an indefinite length. https://github.com/erlang/otp/commit/7f385ebd984ed2931daa761819816b3e9da7d63c /Bjorn On Mon, Sep 22, 2014 at 3:50 PM, Simon Cornish wrote: > Hi Alex, > Your analysis is correct regarding that the underlying fault is > reading beyond the end of the buffer > The attached patch solves your problem in the same (although > stylistically different) way. > It also corrects the fault that a primitive cannot be encoded using an > indefinite length. This is both according to X.690 and common sense! > If I get some breathing room later this week I'll put together a pull > request for this with some test. > Regards, > Simon > > > On 12 September 2014 14:23, Alex Wilson alex-at-cooperi.net > |erlang-mail/gmail| wrote: >> Hi erlang-bugs, >> >> I've found a very interesting segmentation fault bug in >> asn1rt_nif:decode_ber_tlv. >> >> A simple case to reproduce it: just type this at the shell: >> >> asn1rt_nif:decode_ber_tlv(<<"!",16#80>>). >> >> It seems like whether or not the segmentation fault manifests varies a lot >> between different OS, compiler and OTP versions. So far I've reproduced it >> on: >> >> * R16B03-1 on Mac OSX Mavericks (clang) >> * R16B01 on OpenBSD 5.3-stable with gcc 4.2.1 >> * R16B03-1 on Linux (Fedora) with gcc 4.8.3 >> * R17B01 on OpenBSD 5.6-current with gcc 4.2.1 >> * R17 developer build from "maint" branch as of last week, Mac OSX >> Mavericks (clang) >> >> I also asked some random people in #erlang on freenode to try it out and >> they also reproduced the segfault using my test case. >> >> Sometimes it doesn't segfault the first time around, but if you run it a few >> times at the shell it will do it eventually. >> >> >> Backtrace from R17 maint on Mac OSX: >> >> * thread #1: tid = 0x2410f5, 0x00007fff958db1aa >> libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458, queue = >> 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, >> address=0x107ffff0) >> * frame #0: 0x00007fff958db1aa >> libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458 >> frame #1: 0x000000001037aa49 asn1rt_nif.so`decode_ber_tlv_raw [inlined] >> ber_decode_begin(env=0x00007fff5fbff548, in_buf=, >> in_buf_len=, err_pos=) + 80 at asn1_erl_nif.c:854 >> frame #2: 0x000000001037a9f9 >> asn1rt_nif.so`decode_ber_tlv_raw(env=0x00007fff5fbff548, argc=, >> argv=) + 41 at asn1_erl_nif.c:1256 >> frame #3: 0x00000000100e4cbd beam`process_main + 58077 at >> beam_emu.c:3524 >> frame #4: 0x00000000100196dd beam`erl_start(argc=21, argv=) >> + 5997 at erl_init.c:1990 >> frame #5: 0x0000000010000df9 beam`main(argc=, >> argv=) + 9 at erl_main.c:29 >> >> Backtrace from R16B03-1 on Linux (gdb): (no symbols on that machine, sorry, >> but can clearly see it's the same trace) >> >> #0 __memcpy_sse2_unaligned () >> at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:157 >> #1 0x00007f0c48cbdace in ber_decode_begin () >> from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so >> #2 0x00007f0c48cbdb6f in decode_ber_tlv_raw () >> from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so >> #3 0x0000000000533254 in process_main () >> #4 0x00000000004508a0 in erl_start () >> #5 0x0000000000434049 in main () >> >> >> There's a lot of inlining and optimisation going on in this part of the >> code, which makes it hard to look back and forth between the assembly and C. >> >> Anyway, the segfault is caused by running off the end of memory after doing >> a memcpy with -2 as the length (0xfffffffe). This is because ib_index has >> gotten to a value of 4 when the in_buf_len is only 2. >> >> Using some watchpoints I figured out that ber_decode_value's if (indef == 1) >> block is responsible for the incrementing of ib_index beyond the end of the >> binary. I hacked up the following patch: >> >> >> --- a/lib/asn1/c_src/asn1_erl_nif.c >> +++ b/lib/asn1/c_src/asn1_erl_nif.c >> @@ -968,16 +968,16 @@ static int ber_decode_value(ErlNifEnv* env, >> ERL_NIF_TERM *value, unsigned cha >> if (indef == 1) { /* in this case it is desireably to check that >> indefinite length >> end bytes exist in inbuffer */ >> curr_head = enif_make_list(env, 0); >> - while (!(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { >> - if (*ib_index >= in_buf_len) >> - return ASN1_INDEF_LEN_ERROR; >> - >> + while ((*ib_index + 1 < in_buf_len) && >> + !(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { >> if ((maybe_ret = ber_decode(env, &term, in_buf, ib_index, >> in_buf_len)) >> <= ASN1_ERROR >> ) >> return maybe_ret; >> curr_head = enif_make_list_cell(env, term, curr_head); >> } >> + if (*ib_index + 1 >= in_buf_len) >> + return ASN1_INDEF_LEN_ERROR; >> enif_make_reverse_list(env, curr_head, value); >> (*ib_index) += 2; /* skip the indefinite length end bytes */ >> } else if (form == ASN1_CONSTRUCTED) >> >> >> And it seems to stop it happening with my test case on OSX, at least. It >> makes two changes -- checking for *ib_index + 1 >= in_buf_len (because it's >> using in_buf[*ib_index + 1], it should check that _that_ index is valid, not >> just *ib_index). The second change is to move the check from an "if" inside >> the loop to a loop condition. >> >> The movement to the loop condition seems to be the main difference. Looking >> at the assembly, it seems like the compiler is reasoning (after inlining the >> entire ber_decode function into this loop) that it can hoist that if out >> somehow?... maybe because the ber_decode code will already check it (???) >> >> Putting it into the loop condition seems to stop this behaviour. But if I've >> misdiagnosed the problem (quite possible), then perhaps all I've done is >> permute the code just enough to stop the optimiser doing this one thing for >> now, and there are other problems just waiting in the wings... >> >> Hoping somebody who knows the asn1 code better than I do (or C in general) >> might be able to help! >> _______________________________________________ >> erlang-bugs mailing list >> erlang-bugs@REDACTED >> http://erlang.org/mailman/listinfo/erlang-bugs > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From ali.sabil@REDACTED Thu Sep 25 17:07:43 2014 From: ali.sabil@REDACTED (Ali Sabil) Date: Thu, 25 Sep 2014 17:07:43 +0200 Subject: [erlang-bugs] Erlang 17.3: can't load crypto library on OSX Lion Message-ID: Hi all, It looks like the crypto application in Erlang/OTP 17.3 fails on OSX Lion as described here: https://trac.macports.org/ticket/45114 https://github.com/Homebrew/homebrew/pull/32363 Any input on why this would happen? Thanks, Ali -------------- next part -------------- An HTML attachment was scrubbed... URL: From nc@REDACTED Thu Sep 25 18:08:18 2014 From: nc@REDACTED (Nicolas Charpentier) Date: Thu, 25 Sep 2014 12:08:18 -0400 Subject: [erlang-bugs] Erlang 17.3: can't load crypto library on OSX Lion In-Reply-To: References: Message-ID: <9c5b8921bff00cc5a5e3162acbf40872.squirrel@www.geekisp.com> Hi Ali, > Hi all, > > It looks like the crypto application in Erlang/OTP 17.3 fails on OSX Lion > as described here: > https://trac.macports.org/ticket/45114 > https://github.com/Homebrew/homebrew/pull/32363 > > Any input on why this would happen? It might be a problem related to the version of openssl used during the link of crypto.so. I can reproduce it on Maverick. crypto refuses to start with libcrypto.0.9.8 is used, but start correctly with libcrypto.1.0.0 Either it's a compilation problem (mismatch of the header used for the compilation and the lib used to link) or a dependency on openssl 1.0.0 has been introduced in 17.3 You can see the lib dynamique used by crypto.so with dyldinfo -dylibs $ROOT/lib/crypto-3.4/priv/lib/crypto.so When it doesn't work, it depends on /usr/lib/libcrypto.0.9.8.dylib You can change this dependency with ( the example assumes macport to be installed) install_name_tool -change /usr/lib/libcrypto.0.9.8.dylib /opt/local/lib/libcrypto.1.0.0.dylib $ROOT/lib/crypto-3.4/priv/lib/crypto.so Hope it helps /Nicolas From sverker.eriksson@REDACTED Thu Sep 25 20:55:56 2014 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Thu, 25 Sep 2014 20:55:56 +0200 Subject: [erlang-bugs] Erlang 17.3: can't load crypto library on OSX Lion In-Reply-To: <9c5b8921bff00cc5a5e3162acbf40872.squirrel@www.geekisp.com> References: <9c5b8921bff00cc5a5e3162acbf40872.squirrel@www.geekisp.com> Message-ID: <5424653C.5040707@erix.ericsson.se> On 09/25/2014 06:08 PM, Nicolas Charpentier wrote: > Hi Ali, > >> Hi all, >> >> It looks like the crypto application in Erlang/OTP 17.3 fails on OSX Lion >> as described here: >> https://trac.macports.org/ticket/45114 >> https://github.com/Homebrew/homebrew/pull/32363 >> >> Any input on why this would happen? > It might be a problem related to the version of openssl used during the > link of crypto.so. > > I can reproduce it on Maverick. > crypto refuses to start with libcrypto.0.9.8 is used, but start correctly > with libcrypto.1.0.0 > > Either it's a compilation problem (mismatch of the header used for the > compilation and the lib used to link) or a dependency on openssl 1.0.0 has > been introduced in 17.3 > > You can see the lib dynamique used by crypto.so with > dyldinfo -dylibs $ROOT/lib/crypto-3.4/priv/lib/crypto.so > > When it doesn't work, it depends on /usr/lib/libcrypto.0.9.8.dylib > > You can change this dependency with ( the example assumes macport to be > installed) > install_name_tool -change /usr/lib/libcrypto.0.9.8.dylib > /opt/local/lib/libcrypto.1.0.0.dylib > $ROOT/lib/crypto-3.4/priv/lib/crypto.so > > Hope it helps > > /Nicolas > > libcrypto.0.9.* and libcrypto.1.* are not compatible. You can not compile agains one and link against the other. If it seems to work, you are just (un)lucky. In 17.3 the following ticket was fixed to refuse load of crypto with wrong version of libcrypto and thereby avoid strange memory corruption leading to VM crash or other even worse effects. OTP-12146 Make crypto verify major version number of OpenSSL header files and runtime library. Loading of crypto will fail if there is a version mismatch. /Sverker, Erlang/OTP -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Sep 26 04:19:55 2014 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 25 Sep 2014 19:19:55 -0700 Subject: [erlang-bugs] erts_debug:flat_size/1 wrong? Message-ID: <5424CD4B.2090909@gmail.com> Hi, I have been attempting to compare the output of erts_debug:flat_size/1 to the memory info at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923 and the results show that each term's size is off-by-one (at least for pids local/remote, refs local/remote, floats, integers, bignums, binaries and atoms). I know the function is experimental, but this is a bug, right? The problem affects top-level terms and nested terms, so it is likely to understate the memory with large terms. I wanted to make sure the memory info (in the efficiency guide) was accurate (it seems like it is). I was testing with R16B03 on 64bits. For example: 1> erts_debug:flat_size(576460752303423488). 2 2> erts_debug:flat_size(576460752303423487). 0 3> erts_debug:flat_size(undefined). 0 4> erts_debug:flat_size([]). 0 5> erts_debug:flat_size([undefined]). 2 % 1 word for each element in the list * 2 elements including [] 6> erts_debug:flat_size(erlang:make_ref()). 3 7> erts_debug:flat_size(erlang:self()). 0 8> erts_debug:flat_size(1.0). 2 Thanks, Michael From sc@REDACTED Fri Sep 26 07:34:37 2014 From: sc@REDACTED (Sam Chapin) Date: Fri, 26 Sep 2014 01:34:37 -0400 Subject: [erlang-bugs] Funs with duplicate names Message-ID: I'm new to the list, apologies if this is known. Discovered while working in OTP 17.1, and it's still present after updating to 17.3. Create a module containing the following: f1() -> fun F() -> "f1" end(). f2() -> fun F() -> "f2" end(). Load it and execute f2(). On my system, its return value is "f1". -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Fri Sep 26 11:26:11 2014 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Fri, 26 Sep 2014 11:26:11 +0200 Subject: [erlang-bugs] Funs with duplicate names In-Reply-To: References: Message-ID: <21541.12595.885468.8665@gargle.gargle.HOWL> Sam Chapin writes: > I'm new to the list, apologies if this is known. Discovered while working > in OTP 17.1, and it's still present after updating to 17.3. > > Create a module containing the following: > > f1() -> fun F() -> "f1" end(). > > f2() -> fun F() -> "f2" end(). > > Load it and execute f2(). On my system, its return value is "f1". I didn't know about this. Confirmed on my Linux/x86_64 system: > cat bug.erl -module(bug). -compile(export_all). f1() -> fun _F() -> "f1" end. f2() -> fun _F() -> "f2" end. > bin/erl Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.2 (abort with ^G) 1> c(bug). {ok,bug} 2> (bug:f2())(). "f1" 3> R16 and R15 can't compile the code. From n.oxyde@REDACTED Fri Sep 26 11:52:31 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 26 Sep 2014 11:52:31 +0200 Subject: [erlang-bugs] Funs with duplicate names In-Reply-To: <21541.12595.885468.8665@gargle.gargle.HOWL> References: <21541.12595.885468.8665@gargle.gargle.HOWL> Message-ID: <9D9D5B31-8E43-440C-9783-43369C174F48@gmail.com> That's damn horrible. The bug probably occurs in v3_core given that recursive funs which don't recursive are compiled to plain old functions. As an aside, I am more and more of the opinion that compilers should proven correct, not tested. -- Anthony Ramine > Le 26 sept. 2014 ? 11:26, Mikael Pettersson a ?crit : > > Sam Chapin writes: >> I'm new to the list, apologies if this is known. Discovered while working >> in OTP 17.1, and it's still present after updating to 17.3. >> >> Create a module containing the following: >> >> f1() -> fun F() -> "f1" end(). >> >> f2() -> fun F() -> "f2" end(). >> >> Load it and execute f2(). On my system, its return value is "f1". > > I didn't know about this. Confirmed on my Linux/x86_64 system: > >> cat bug.erl > -module(bug). > -compile(export_all). > f1() -> fun _F() -> "f1" end. > f2() -> fun _F() -> "f2" end. >> bin/erl > Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V6.2 (abort with ^G) > 1> c(bug). > {ok,bug} > 2> (bug:f2())(). > "f1" > 3> > > R16 and R15 can't compile the code. > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs From n.oxyde@REDACTED Fri Sep 26 12:07:29 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 26 Sep 2014 12:07:29 +0200 Subject: [erlang-bugs] Funs with duplicate names In-Reply-To: <9D9D5B31-8E43-440C-9783-43369C174F48@gmail.com> References: <21541.12595.885468.8665@gargle.gargle.HOWL> <9D9D5B31-8E43-440C-9783-43369C174F48@gmail.com> Message-ID: Or not. It happens later, because the compiler is confused by both functions donning {'function_name',{'_F',0}} as their annotation. Le 26 sept. 2014 ? 11:52, Anthony Ramine a ?crit : > v3_core From sverker.eriksson@REDACTED Fri Sep 26 15:46:44 2014 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 26 Sep 2014 15:46:44 +0200 Subject: [erlang-bugs] erts_debug:flat_size/1 wrong? In-Reply-To: <5424CD4B.2090909@gmail.com> References: <5424CD4B.2090909@gmail.com> Message-ID: <54256E44.1080305@erix.ericsson.se> On 09/26/2014 04:19 AM, Michael Truog wrote: > Hi, > > I have been attempting to compare the output of erts_debug:flat_size/1 > to the memory info at > http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923 and > the results show that each term's size is off-by-one (at least for > pids local/remote, refs local/remote, floats, integers, bignums, > binaries and atoms). I know the function is experimental, but this is > a bug, right? The problem affects top-level terms and nested terms, > so it is likely to understate the memory with large terms. I wanted > to make sure the memory info (in the efficiency guide) was accurate > (it seems like it is). I was testing with R16B03 on 64bits. > > For example: > 1> erts_debug:flat_size(576460752303423488). > 2 > 2> erts_debug:flat_size(576460752303423487). > 0 > 3> erts_debug:flat_size(undefined). > 0 > 4> erts_debug:flat_size([]). > 0 > 5> erts_debug:flat_size([undefined]). > 2 > % 1 word for each element in the list * 2 elements including [] > 6> erts_debug:flat_size(erlang:make_ref()). > 3 > 7> erts_debug:flat_size(erlang:self()). > 0 > 8> erts_debug:flat_size(1.0). > 2 > erts_debug:flat_size/1 is undocumented and can therefor not be wrong by definition :-) But no, there is no bug, erts_debug:flat_size/1 works as intended in the current implementation. It returns the number of words on the _heap_ occupied (*) by the term. Excluded are thus off-heap data such as binaries larger than 64 bytes AND the top term word, which is kept in a register or on the stack. (*) erts_debug:flat_size does not take sharing of sub-terms into account while erts_debug:size/1 do. 1> A = "Hello". "Hello" 2> erts_debug:flat_size(A). 10 3> erts_debug:flat_size({A,A}). 23 4> erts_debug:size({A,A}). 13 /Sverker, Erlang/OTP From mjtruog@REDACTED Fri Sep 26 20:27:47 2014 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 26 Sep 2014 11:27:47 -0700 Subject: [erlang-bugs] erts_debug:flat_size/1 wrong? In-Reply-To: <54256E44.1080305@erix.ericsson.se> References: <5424CD4B.2090909@gmail.com> <54256E44.1080305@erix.ericsson.se> Message-ID: <5425B023.6080906@gmail.com> On 09/26/2014 06:46 AM, Sverker Eriksson wrote: > On 09/26/2014 04:19 AM, Michael Truog wrote: >> Hi, >> >> I have been attempting to compare the output of erts_debug:flat_size/1 to the memory info at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923 and the results show that each term's size is off-by-one (at least for pids local/remote, refs local/remote, floats, integers, bignums, binaries and atoms). I know the function is experimental, but this is a bug, right? The problem affects top-level terms and nested terms, so it is likely to understate the memory with large terms. I wanted to make sure the memory info (in the efficiency guide) was accurate (it seems like it is). I was testing with R16B03 on 64bits. >> >> For example: >> 1> erts_debug:flat_size(576460752303423488). >> 2 >> 2> erts_debug:flat_size(576460752303423487). >> 0 >> 3> erts_debug:flat_size(undefined). >> 0 >> 4> erts_debug:flat_size([]). >> 0 >> 5> erts_debug:flat_size([undefined]). >> 2 >> % 1 word for each element in the list * 2 elements including [] >> 6> erts_debug:flat_size(erlang:make_ref()). >> 3 >> 7> erts_debug:flat_size(erlang:self()). >> 0 >> 8> erts_debug:flat_size(1.0). >> 2 >> > > > erts_debug:flat_size/1 is undocumented and can therefor not be wrong by definition :-) > > But no, there is no bug, erts_debug:flat_size/1 works as intended in the current implementation. It returns the number of words on the _heap_ occupied (*) by the term. Excluded are thus off-heap data such as binaries larger than 64 bytes AND the top term word, which is kept in a register or on the stack. > > (*) erts_debug:flat_size does not take sharing of sub-terms into account while erts_debug:size/1 do. > > 1> A = "Hello". > "Hello" > 2> erts_debug:flat_size(A). > 10 > 3> erts_debug:flat_size({A,A}). > 23 > 4> erts_debug:size({A,A}). > 13 Thank you for the information. However, based on this the Efficiency Guide memory information should be slightly wrong (at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923), since the information should be * List: 1 word + |2 word per element| + the size of each element * Tuple: 2 words + |1 word per element| + the size of each element and not what it currently is: * List: 1 word + |1 word per element| + the size of each element * Tuple: 2 words + the size of each element That is at least what I have found when using erts_debug:flat_size/1 in the repo https://github.com/okeuday/erlang_term when looking at the total size of an Erlang term. Do you agree? Thanks, Michael > > > /Sverker, Erlang/OTP > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zl9d97p02@REDACTED Fri Sep 26 23:43:53 2014 From: zl9d97p02@REDACTED (Simon Cornish) Date: Fri, 26 Sep 2014 14:43:53 -0700 Subject: [erlang-bugs] Segfault in asn1rt_nif:decode_ber_tlv In-Reply-To: References: <5412E5A6.7020202@cooperi.net> <20023-1411393813-601401@sneakemail.com> Message-ID: <31534-1411767833-977237@sneakemail.com> Hi Bj?rn, Do you intend to add that test to a later release? Whilst I could not crash the VM without it, it's not possible to unambiguously decode some primitives with indefinite length (eg. OCTET STRING, INTEGER, etc). I'm all for strict encoding and more permissive decoders but in this case I see more benefit in raising an error. Regards, Simon On 24 September 2014 07:11, Bj?rn Gustavsson bjorn-at-erlang.org |erlang-mail/gmail| <394upzd31t@REDACTED> wrote: > Hi Simon, > > As you can see in my late answer to Alex, > we have fixed the bug. But I did not add > an test for a primitive type encoded with > an indefinite length. > > https://github.com/erlang/otp/commit/7f385ebd984ed2931daa761819816b3e9da7d63c > > /Bjorn > > On Mon, Sep 22, 2014 at 3:50 PM, Simon Cornish wrote: >> Hi Alex, >> Your analysis is correct regarding that the underlying fault is >> reading beyond the end of the buffer >> The attached patch solves your problem in the same (although >> stylistically different) way. >> It also corrects the fault that a primitive cannot be encoded using an >> indefinite length. This is both according to X.690 and common sense! >> If I get some breathing room later this week I'll put together a pull >> request for this with some test. >> Regards, >> Simon >> >> >> On 12 September 2014 14:23, Alex Wilson alex-at-cooperi.net >> |erlang-mail/gmail| wrote: >>> Hi erlang-bugs, >>> >>> I've found a very interesting segmentation fault bug in >>> asn1rt_nif:decode_ber_tlv. >>> >>> A simple case to reproduce it: just type this at the shell: >>> >>> asn1rt_nif:decode_ber_tlv(<<"!",16#80>>). >>> >>> It seems like whether or not the segmentation fault manifests varies a lot >>> between different OS, compiler and OTP versions. So far I've reproduced it >>> on: >>> >>> * R16B03-1 on Mac OSX Mavericks (clang) >>> * R16B01 on OpenBSD 5.3-stable with gcc 4.2.1 >>> * R16B03-1 on Linux (Fedora) with gcc 4.8.3 >>> * R17B01 on OpenBSD 5.6-current with gcc 4.2.1 >>> * R17 developer build from "maint" branch as of last week, Mac OSX >>> Mavericks (clang) >>> >>> I also asked some random people in #erlang on freenode to try it out and >>> they also reproduced the segfault using my test case. >>> >>> Sometimes it doesn't segfault the first time around, but if you run it a few >>> times at the shell it will do it eventually. >>> >>> >>> Backtrace from R17 maint on Mac OSX: >>> >>> * thread #1: tid = 0x2410f5, 0x00007fff958db1aa >>> libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458, queue = >>> 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, >>> address=0x107ffff0) >>> * frame #0: 0x00007fff958db1aa >>> libsystem_platform.dylib`_platform_memmove$VARIANT$Nehalem + 458 >>> frame #1: 0x000000001037aa49 asn1rt_nif.so`decode_ber_tlv_raw [inlined] >>> ber_decode_begin(env=0x00007fff5fbff548, in_buf=, >>> in_buf_len=, err_pos=) + 80 at asn1_erl_nif.c:854 >>> frame #2: 0x000000001037a9f9 >>> asn1rt_nif.so`decode_ber_tlv_raw(env=0x00007fff5fbff548, argc=, >>> argv=) + 41 at asn1_erl_nif.c:1256 >>> frame #3: 0x00000000100e4cbd beam`process_main + 58077 at >>> beam_emu.c:3524 >>> frame #4: 0x00000000100196dd beam`erl_start(argc=21, argv=) >>> + 5997 at erl_init.c:1990 >>> frame #5: 0x0000000010000df9 beam`main(argc=, >>> argv=) + 9 at erl_main.c:29 >>> >>> Backtrace from R16B03-1 on Linux (gdb): (no symbols on that machine, sorry, >>> but can clearly see it's the same trace) >>> >>> #0 __memcpy_sse2_unaligned () >>> at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:157 >>> #1 0x00007f0c48cbdace in ber_decode_begin () >>> from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so >>> #2 0x00007f0c48cbdb6f in decode_ber_tlv_raw () >>> from /usr/lib64/erlang/lib/asn1-2.0.4/priv/lib/asn1_erl_nif.so >>> #3 0x0000000000533254 in process_main () >>> #4 0x00000000004508a0 in erl_start () >>> #5 0x0000000000434049 in main () >>> >>> >>> There's a lot of inlining and optimisation going on in this part of the >>> code, which makes it hard to look back and forth between the assembly and C. >>> >>> Anyway, the segfault is caused by running off the end of memory after doing >>> a memcpy with -2 as the length (0xfffffffe). This is because ib_index has >>> gotten to a value of 4 when the in_buf_len is only 2. >>> >>> Using some watchpoints I figured out that ber_decode_value's if (indef == 1) >>> block is responsible for the incrementing of ib_index beyond the end of the >>> binary. I hacked up the following patch: >>> >>> >>> --- a/lib/asn1/c_src/asn1_erl_nif.c >>> +++ b/lib/asn1/c_src/asn1_erl_nif.c >>> @@ -968,16 +968,16 @@ static int ber_decode_value(ErlNifEnv* env, >>> ERL_NIF_TERM *value, unsigned cha >>> if (indef == 1) { /* in this case it is desireably to check that >>> indefinite length >>> end bytes exist in inbuffer */ >>> curr_head = enif_make_list(env, 0); >>> - while (!(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { >>> - if (*ib_index >= in_buf_len) >>> - return ASN1_INDEF_LEN_ERROR; >>> - >>> + while ((*ib_index + 1 < in_buf_len) && >>> + !(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) { >>> if ((maybe_ret = ber_decode(env, &term, in_buf, ib_index, >>> in_buf_len)) >>> <= ASN1_ERROR >>> ) >>> return maybe_ret; >>> curr_head = enif_make_list_cell(env, term, curr_head); >>> } >>> + if (*ib_index + 1 >= in_buf_len) >>> + return ASN1_INDEF_LEN_ERROR; >>> enif_make_reverse_list(env, curr_head, value); >>> (*ib_index) += 2; /* skip the indefinite length end bytes */ >>> } else if (form == ASN1_CONSTRUCTED) >>> >>> >>> And it seems to stop it happening with my test case on OSX, at least. It >>> makes two changes -- checking for *ib_index + 1 >= in_buf_len (because it's >>> using in_buf[*ib_index + 1], it should check that _that_ index is valid, not >>> just *ib_index). The second change is to move the check from an "if" inside >>> the loop to a loop condition. >>> >>> The movement to the loop condition seems to be the main difference. Looking >>> at the assembly, it seems like the compiler is reasoning (after inlining the >>> entire ber_decode function into this loop) that it can hoist that if out >>> somehow?... maybe because the ber_decode code will already check it (???) >>> >>> Putting it into the loop condition seems to stop this behaviour. But if I've >>> misdiagnosed the problem (quite possible), then perhaps all I've done is >>> permute the code just enough to stop the optimiser doing this one thing for >>> now, and there are other problems just waiting in the wings... >>> >>> Hoping somebody who knows the asn1 code better than I do (or C in general) >>> might be able to help! >>> _______________________________________________ >>> erlang-bugs mailing list >>> erlang-bugs@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-bugs >> _______________________________________________ >> erlang-bugs mailing list >> erlang-bugs@REDACTED >> http://erlang.org/mailman/listinfo/erlang-bugs > > > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From wieslaw.bieniek@REDACTED Fri Sep 26 15:03:47 2014 From: wieslaw.bieniek@REDACTED (=?UTF-8?B?V2llc8WCYXcgQmllbmllaw==?=) Date: Fri, 26 Sep 2014 15:03:47 +0200 Subject: [erlang-bugs] Why ets called ac_tab is public ? Message-ID: <54256433.1060308@comarch.pl> Hello, I was wondering why the ets holding application envs is public : ets:new(ac_tab, [set, public, named_table]) (from application_controller) ? Reading of envs is done by plain ets reads and setting envs is done by call to gen_server. So it seems protected access shuold be enough. My question is: Why is it public ? Thanks in advance -- *Wies?aw Bieniek* Designer Telco BSS R&D tel. +48 12 646 12 66 website: www.comarch.pl -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2197 bytes Desc: Kryptograficzna sygnatura S/MIME URL: From lenrel2014@REDACTED Fri Sep 26 16:26:28 2014 From: lenrel2014@REDACTED (Lenrel Lenrel) Date: Fri, 26 Sep 2014 16:26:28 +0200 Subject: [erlang-bugs] beam.smp hangs and use 100% of cpu Message-ID: I have compiled from source erlang 17.3 and rabbitmq 3.3.5. Then I generate a load of aprox. 200 msg/s to rabbitmq exchange. After a couple of minutes, process beam.smp hangs and use 100% of cpu. The rabbitmq stops receiving connections and can't do anything with rabbitmqctl. The same thing happens when I compile older versions of erlang and rabbitmq (R16B03 / 3.1.5). This is the strafe -f -p: Process 3431 attached with 39 threads [pid 3509] futex(0x7fa289380990, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3510] futex(0x7fa2893809d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3508] futex(0x7fa289380950, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3506] futex(0x7fa2893808d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3507] wait4(-1, [pid 3505] futex(0x7fa289380890, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3504] futex(0x7fa289380850, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3503] futex(0x7fa289380810, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3502] futex(0x7fa2893807d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3501] futex(0x7fa289380790, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3500] futex(0x7fa289380750, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3499] futex(0x7fa289380710, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3497] futex(0x7fa289380690, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3496] futex(0x7fa289380650, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3498] futex(0x7fa2893806d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3495] futex(0x7fa289380610, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3494] futex(0x7fa2893805d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3493] futex(0x7fa289380590, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3492] futex(0x7fa289380550, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3491] futex(0x7fa289380510, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3490] futex(0x7fa2893804d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3489] futex(0x7fa289380490, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3488] futex(0x7fa289380450, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3487] futex(0x7fa289380410, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3486] futex(0x7fa2893803d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3485] futex(0x7fa289380390, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3484] futex(0x7fa289380350, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3483] futex(0x7fa289380310, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3482] futex(0x7fa2893802d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3481] futex(0x7fa289380290, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3480] futex(0x7fa289380250, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3479] futex(0x7fa289380210, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3478] futex(0x7fa2893801d0, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3477] futex(0x7fa289380190, FUTEX_WAIT_PRIVATE, 4294967295, NULL [pid 3476] futex(0x8dbd44, FUTEX_WAIT_PRIVATE, 3, NULL [pid 3475] read(6, [pid 3431] select(0, NULL, NULL, NULL, NULL any help? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Sep 27 15:29:39 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 27 Sep 2014 15:29:39 +0200 Subject: [erlang-bugs] beam.smp hangs and use 100% of cpu In-Reply-To: References: Message-ID: On Fri, Sep 26, 2014 at 4:26 PM, Lenrel Lenrel wrote: > I have compiled from source erlang 17.3 and rabbitmq 3.3.5. We need more information: System architecture / operating system What compilation options were used Do you use HiPE? arguments to the VM Can you log in to the console on the node? Do you have a crashdump? (I believe sending SIGQUIT to the beam.smp process will produce one, but do note environment variables in the erl(1) man page). -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Sat Sep 27 15:40:43 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 27 Sep 2014 15:40:43 +0200 Subject: [erlang-bugs] Funs with duplicate names In-Reply-To: References: <21541.12595.885468.8665@gargle.gargle.HOWL> <9D9D5B31-8E43-440C-9783-43369C174F48@gmail.com> Message-ID: <48FC3BB1-386C-453A-A498-659FC34DAB1E@gmail.com> https://github.com/erlang/otp/pull/485 Le 26 sept. 2014 ? 12:07, Anthony Ramine a ?crit : > Or not. > > It happens later, because the compiler is confused by both functions donning {'function_name',{'_F',0}} as their annotation. > > Le 26 sept. 2014 ? 11:52, Anthony Ramine a ?crit : > >> v3_core > From rvirding@REDACTED Sun Sep 28 09:00:50 2014 From: rvirding@REDACTED (Robert Virding) Date: Sun, 28 Sep 2014 00:00:50 -0700 Subject: [erlang-bugs] erts_debug:flat_size/1 wrong? In-Reply-To: <5425B023.6080906@gmail.com> References: <5424CD4B.2090909@gmail.com> <54256E44.1080305@erix.ericsson.se> <5425B023.6080906@gmail.com> Message-ID: Perhaps an easy solution would be to document, unofficially of course so it is not doucmented, exactly how erts_debug:flat_size/1 works. Robert On 26 September 2014 11:27, Michael Truog wrote: > On 09/26/2014 06:46 AM, Sverker Eriksson wrote: > > On 09/26/2014 04:19 AM, Michael Truog wrote: > > Hi, > > I have been attempting to compare the output of erts_debug:flat_size/1 to > the memory info at > http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923 and the > results show that each term's size is off-by-one (at least for pids > local/remote, refs local/remote, floats, integers, bignums, binaries and > atoms). I know the function is experimental, but this is a bug, right? > The problem affects top-level terms and nested terms, so it is likely to > understate the memory with large terms. I wanted to make sure the memory > info (in the efficiency guide) was accurate (it seems like it is). I was > testing with R16B03 on 64bits. > > For example: > 1> erts_debug:flat_size(576460752303423488). > 2 > 2> erts_debug:flat_size(576460752303423487). > 0 > 3> erts_debug:flat_size(undefined). > 0 > 4> erts_debug:flat_size([]). > 0 > 5> erts_debug:flat_size([undefined]). > 2 > % 1 word for each element in the list * 2 elements including [] > 6> erts_debug:flat_size(erlang:make_ref()). > 3 > 7> erts_debug:flat_size(erlang:self()). > 0 > 8> erts_debug:flat_size(1.0). > 2 > > > > erts_debug:flat_size/1 is undocumented and can therefor not be wrong by > definition :-) > > But no, there is no bug, erts_debug:flat_size/1 works as intended in the > current implementation. It returns the number of words on the _heap_ > occupied (*) by the term. Excluded are thus off-heap data such as binaries > larger than 64 bytes AND the top term word, which is kept in a register or > on the stack. > > (*) erts_debug:flat_size does not take sharing of sub-terms into account > while erts_debug:size/1 do. > > 1> A = "Hello". > "Hello" > 2> erts_debug:flat_size(A). > 10 > 3> erts_debug:flat_size({A,A}). > 23 > 4> erts_debug:size({A,A}). > 13 > > > Thank you for the information. However, based on this the Efficiency > Guide memory information should be slightly wrong (at > http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923), since > the information should be > * List: 1 word + 2 word per element + the size of each element > * Tuple: 2 words + 1 word per element + the size of each element > > and not what it currently is: > * List: 1 word + 1 word per element + the size of each element > * Tuple: 2 words + the size of each element > > That is at least what I have found when using erts_debug:flat_size/1 in > the repo https://github.com/okeuday/erlang_term when looking at the total > size of an Erlang term. Do you agree? > > Thanks, > Michael > > > > /Sverker, Erlang/OTP > > > > > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Sun Sep 28 14:28:18 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sun, 28 Sep 2014 14:28:18 +0200 Subject: [erlang-bugs] erts_debug:flat_size/1 wrong? In-Reply-To: References: <5424CD4B.2090909@gmail.com> <54256E44.1080305@erix.ericsson.se> <5425B023.6080906@gmail.com> Message-ID: <7F8A7DC9-71DD-46BF-B74A-47EDA052EE7C@gmail.com> While we are at it, I would like erts_debug to be a proper module, with a module_info function etc. Le 28 sept. 2014 ? 09:00, Robert Virding a ?crit : > Perhaps an easy solution would be to document, unofficially of course so it is not doucmented, exactly how erts_debug:flat_size/1 works. > > Robert > > > On 26 September 2014 11:27, Michael Truog wrote: > On 09/26/2014 06:46 AM, Sverker Eriksson wrote: >> On 09/26/2014 04:19 AM, Michael Truog wrote: >>> Hi, >>> >>> I have been attempting to compare the output of erts_debug:flat_size/1 to the memory info at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923 and the results show that each term's size is off-by-one (at least for pids local/remote, refs local/remote, floats, integers, bignums, binaries and atoms). I know the function is experimental, but this is a bug, right? The problem affects top-level terms and nested terms, so it is likely to understate the memory with large terms. I wanted to make sure the memory info (in the efficiency guide) was accurate (it seems like it is). I was testing with R16B03 on 64bits. >>> >>> For example: >>> 1> erts_debug:flat_size(576460752303423488). >>> 2 >>> 2> erts_debug:flat_size(576460752303423487). >>> 0 >>> 3> erts_debug:flat_size(undefined). >>> 0 >>> 4> erts_debug:flat_size([]). >>> 0 >>> 5> erts_debug:flat_size([undefined]). >>> 2 >>> % 1 word for each element in the list * 2 elements including [] >>> 6> erts_debug:flat_size(erlang:make_ref()). >>> 3 >>> 7> erts_debug:flat_size(erlang:self()). >>> 0 >>> 8> erts_debug:flat_size(1.0). >>> 2 >>> >> >> >> erts_debug:flat_size/1 is undocumented and can therefor not be wrong by definition :-) >> >> But no, there is no bug, erts_debug:flat_size/1 works as intended in the current implementation. It returns the number of words on the _heap_ occupied (*) by the term. Excluded are thus off-heap data such as binaries larger than 64 bytes AND the top term word, which is kept in a register or on the stack. >> >> (*) erts_debug:flat_size does not take sharing of sub-terms into account while erts_debug:size/1 do. >> >> 1> A = "Hello". >> "Hello" >> 2> erts_debug:flat_size(A). >> 10 >> 3> erts_debug:flat_size({A,A}). >> 23 >> 4> erts_debug:size({A,A}). >> 13 > > Thank you for the information. However, based on this the Efficiency Guide memory information should be slightly wrong (at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923), since the information should be > * List: 1 word + 2 word per element + the size of each element > * Tuple: 2 words + 1 word per element + the size of each element > > and not what it currently is: > * List: 1 word + 1 word per element + the size of each element > * Tuple: 2 words + the size of each element > > That is at least what I have found when using erts_debug:flat_size/1 in the repo https://github.com/okeuday/erlang_term when looking at the total size of an Erlang term. Do you agree? > > Thanks, > Michael >> >> >> /Sverker, Erlang/OTP >> >> >> >> > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs From bjorn@REDACTED Mon Sep 29 13:22:47 2014 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 29 Sep 2014 13:22:47 +0200 Subject: [erlang-bugs] Segfault in asn1rt_nif:decode_ber_tlv In-Reply-To: <31534-1411767833-977237@sneakemail.com> References: <5412E5A6.7020202@cooperi.net> <20023-1411393813-601401@sneakemail.com> <31534-1411767833-977237@sneakemail.com> Message-ID: On Fri, Sep 26, 2014 at 11:43 PM, Simon Cornish wrote: > Hi Bj?rn, > Do you intend to add that test to a later release? Whilst I could not > crash the VM without it, it's not possible to unambiguously decode > some primitives with indefinite length (eg. OCTET STRING, INTEGER, > etc). Yes, we will add that test in the next release. Here is the pull request for my fix: https://github.com/erlang/otp/pull/487 /Bjorn From mjtruog@REDACTED Mon Sep 29 16:09:35 2014 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 29 Sep 2014 07:09:35 -0700 Subject: [erlang-bugs] erts_debug:flat_size/1 wrong? In-Reply-To: <5425B023.6080906@gmail.com> References: <5424CD4B.2090909@gmail.com> <54256E44.1080305@erix.ericsson.se> <5425B023.6080906@gmail.com> Message-ID: <5429681F.9020302@gmail.com> On 09/26/2014 11:27 AM, Michael Truog wrote: > On 09/26/2014 06:46 AM, Sverker Eriksson wrote: >> On 09/26/2014 04:19 AM, Michael Truog wrote: >>> Hi, >>> >>> I have been attempting to compare the output of erts_debug:flat_size/1 to the memory info at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923 and the results show that each term's size is off-by-one (at least for pids local/remote, refs local/remote, floats, integers, bignums, binaries and atoms). I know the function is experimental, but this is a bug, right? The problem affects top-level terms and nested terms, so it is likely to understate the memory with large terms. I wanted to make sure the memory info (in the efficiency guide) was accurate (it seems like it is). I was testing with R16B03 on 64bits. >>> >>> For example: >>> 1> erts_debug:flat_size(576460752303423488). >>> 2 >>> 2> erts_debug:flat_size(576460752303423487). >>> 0 >>> 3> erts_debug:flat_size(undefined). >>> 0 >>> 4> erts_debug:flat_size([]). >>> 0 >>> 5> erts_debug:flat_size([undefined]). >>> 2 >>> % 1 word for each element in the list * 2 elements including [] >>> 6> erts_debug:flat_size(erlang:make_ref()). >>> 3 >>> 7> erts_debug:flat_size(erlang:self()). >>> 0 >>> 8> erts_debug:flat_size(1.0). >>> 2 >>> >> >> >> erts_debug:flat_size/1 is undocumented and can therefor not be wrong by definition :-) >> >> But no, there is no bug, erts_debug:flat_size/1 works as intended in the current implementation. It returns the number of words on the _heap_ occupied (*) by the term. Excluded are thus off-heap data such as binaries larger than 64 bytes AND the top term word, which is kept in a register or on the stack. >> >> (*) erts_debug:flat_size does not take sharing of sub-terms into account while erts_debug:size/1 do. >> >> 1> A = "Hello". >> "Hello" >> 2> erts_debug:flat_size(A). >> 10 >> 3> erts_debug:flat_size({A,A}). >> 23 >> 4> erts_debug:size({A,A}). >> 13 > > Thank you for the information. However, based on this the Efficiency Guide memory information should be slightly wrong (at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923), since the information should be > * List: 1 word + |2 word per element| + the size of each element > * Tuple: 2 words + |1 word per element| + the size of each element > > and not what it currently is: > * List: 1 word + |1 word per element| + the size of each element > * Tuple: 2 words + the size of each element > > That is at least what I have found when using erts_debug:flat_size/1 in the repo https://github.com/okeuday/erlang_term when looking at the total size of an Erlang term. Do you agree? After looking at this more I have realized the documentation of the memory information is correct as would be expected. Sorry for the noise about this. Some comment that talks about erts_debug:flat_size/1 (and erts_debug:size/1) providing the process heap size only, with an additional 1 word excluded for the register or stack storage of the top-level term would help make things clearer. This may be straight-forward for some since it makes logical sense, but I didn't know about these internal details and I wanted to be sure I was looking at the size correctly. Thanks, Michael > > Thanks, > Michael >> >> >> /Sverker, Erlang/OTP >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pabftk@REDACTED Mon Sep 29 11:29:54 2014 From: pabftk@REDACTED (Pavel Baturko) Date: Mon, 29 Sep 2014 13:29:54 +0400 Subject: [erlang-bugs] bind_to is ignored in snmpm_net_if Message-ID: Hi, I'm looking into erlang snmp manager code because my manager stops working when I updated erlang from 17.1 to 17.3. The problem is that in otp/lib/snmp/src/manager/snmpm_net_if.erl (github erlang/otp, branch maint) in function socket_params parameter BindTo is ignored in case of Family==inet and init :get_argument(snmp_fd)==error: socket_params(Domain, {IpAddr, IpPort}, BindTo, CommonSocketOpts) -> ... case Family of inet -> case init:get_argument(snmp_fd) of {ok, [[FdStr]]} -> ... error -> {IpPort, [{ip, IpAddr} | SocketOpts]} <<<<<<<< * end; _ -> ... end. *: here ip option is added regardless of BindTo argument. in other cases branches this option is utilized. When address option is not specified in manager.conf snmp manager uses 127.0.0.1 as default and socket is binding to 127.0.0.1. After that all gen_upd:send fails with einval. Thanks, Pavel -------------- next part -------------- An HTML attachment was scrubbed... URL: From liu1985629@REDACTED Mon Sep 29 12:22:21 2014 From: liu1985629@REDACTED (=?GBK?B?wfXQobfJ?=) Date: Mon, 29 Sep 2014 18:22:21 +0800 (CST) Subject: [erlang-bugs] Erlang vm beam.smp crash Message-ID: <6a7d6a4b.104f6.148c0eeb097.Coremail.liu1985629@163.com> I use http://www.erlang.org/download/otp_src_17.0.tar.gz to build the erlang. BIF_RETTYPE port_get_data_1(BIF_ALIST_1) { /* * This is not a signal. See comment above. */ Eterm res; erts_aint_t data; Port* prt; prt = data_lookup_port(BIF_P, BIF_ARG_1); if (!prt) BIF_ERROR(BIF_P, BADARG); data = erts_smp_atomic_read_ddrb(&prt->data); if (!data) BIF_ERROR(BIF_P, BADARG); //I add the two lines to correct it. if ((data & 0x3) != 0) { res = (Eterm) (UWord) data; ASSERT(is_immed(res)); } else { ErtsPortDataHeap *pdhp = (ErtsPortDataHeap *) data; Eterm *hp = HAlloc(BIF_P, pdhp->hsize); res = copy_struct(pdhp->data, pdhp->hsize, &hp, &MSO(BIF_P)); } BIF_RET(res); } (gdb) bt full #0 0x0000000000514524 in port_get_data_1 (A__p=0x7f4bc0d66488, BIF__ARGS=) at beam/erl_bif_port.c:591 pdhp = 0x0 hp = data = 0 #1 0x000000000054d517 in process_main () at beam/beam_emu.c:2787 bf = 0x514490 result = 1688368833101607 init_done = 1 c_p = 0x7f4bc0d66488 reds_used = 178536832 x0 = 1688368833101607 reg = 0x7f4c0aa44180 HTOP = 0x7f4bc036d350 E = 0x7f4bc0370b18 I = 0x7f4bfb5c7af8 FCALLS = 1984 tmp_arg1 = 139963324058344 tmp_arg2 = 15 tmp_big = {139964436718400, 5662828} freg = 0x7f4c0aa461c0 neg_o_reds = 0 arith_func = 0 opcodes = {0x54c14a, 0x54b78e, 0x54c06a, 0x54c0eb, 0x54c2f8, 0x54cee5, 0x54cb67, 0x54e173, 0x54ec5c, 0x54ca4d, 0x54ca43, 0x54ca23, 0x54908b, 0x54c5fe, 0x54d5c8, 0x54d605, 0x54d5f6, 0x54957d, 0x549451, 0x54d366, 0x54d26d, 0x54d29b, 0x54d063, 0x54d092, 0x54d245, 0x54d223, 0x54d176, 0x54d4bb, 0x54ca52, 0x54caa1, 0x54ba31, 0x54ba07, 0x54ba26, 0x54bf45, 0x54bf66, 0x54ccfb, 0x54c949, 0x546667, 0x54ca9c, 0x54674e, 0x546771, 0x54667a, 0x5466ab, 0x5466dc, 0x546715, 0x5464cd, 0x54eaa8, 0x54e14e, 0x54ea52, 0x54eb16, 0x546795, 0x5467b6, 0x5467d8, 0x5467f5, 0x546823, 0x546852, 0x546870, 0x54689f, 0x5468cf, 0x5468fc, 0x54692a, 0x546957, 0x54699e, 0x5469e6, 0x546a14, 0x546a5c, 0x546aa5, 0x546ad3, 0x546b02, 0x546b30, 0x546b78, 0x546bc1, 0x546bf0, 0x546c39, 0x54d3bf, 0x54b2ce, 0x54b39e, 0x54b3c9, 0x54e468, 0x54b3bf, 0x54b5f8, 0x54e1d9, 0x54b046, 0x54b0ac, 0x54b65f, 0x54ce89, 0x54b47a, 0x54d44b, 0x54e23b, 0x54b4bd, 0x5493b7, 0x54947d, 0x54ddf7, 0x54df22, 0x54e011, 0x54d8dd, 0x54d60f, 0x54d694, 0x54d965, 0x54d9db, 0x54da59, 0x54db67, 0x54d718, 0x54d348, 0x54d2c2, 0x54d340, 0x54b103, 0x54d34d, 0x54d1a0, 0x54d8d6, 0x54d857, 0x54d8b7, 0x54d6ad, 0x54d57d, 0x54d5ba, 0x54d810, 0x54d849, 0x549430, 0x54bbbe, 0x54e2f0, 0x54d38c, 0x54e331, 0x54e345, 0x54bab9, 0x54e357, 0x549185, 0x54e3c9, 0x54e3e9, 0x5494f1, 0x549306, 0x549515, 0x54d6a5, 0x54bbc9, 0x54953a, 0x54d0ee, 0x54d0b2, 0x54df04, 0x54ddd1, 0x54bc6c, 0x54dede, 0x54dd60, 0x54dc7d, 0x54dcef, 0x54af53, 0x54afcc, 0x54e7ca, 0x54e7ff, 0x54e294, 0x54e2be, 0x54b6ca, 0x54d355, 0x54d211, 0x54cf47, 0x54cf9a, 0x54d007, 0x54d12e, 0x54ec20, 0x54b559, 0x54b507, 0x546539, 0x54656a, 0x5465a3, 0x5465e0, 0x54e0ea, 0x54e400, 0x546504, 0x5464e2, 0x54ea08, 0x54be99, 0x54c53b, 0x54c5d7, 0x54e774, 0x54bee0, 0x54bf28, 0x54bf36, 0x5464cd, 0x54b222, 0x546d1f, 0x546d3e, 0x546d81, 0x546da0, 0x546dd2, 0x546e29, 0x546e49, 0x546e7c, 0x546d04, 0x546d5e, 0x546e05, 0x546ca2, 0x546cbd, 0x546ce0, 0x546c83, 0x54e83c, 0x54b1cc, 0x54b278, 0x54eb5b, 0x54ba46, 0x54c683, 0x54c726, 0x54c7b4...} temp_bits = 139964436760704 pt_arity = 139963334550664 start_time = 0 start_time_i = 0x0 EBS = 0x7f4c0288c898 #2 0x00000000004a081b in sched_thread_func (vesdp=0x7f4c0288c880) at beam/erl_process.c:7665 callbacks = {arg = 0x7f4c02882380, wakeup = 0x4a21b0 , prepare_wait = 0x49e370 , wait = 0x49f6f0 , finalize_wait = 0x49e350 } esdp = 0x7f4c0288c880 no = 2 #3 0x00000000005df676 in thr_wrapper (vtwd=) at pthread/ethread.c:110 result = res = 0x7fffd82d4b90 twd = thr_func = 0x4a0700 arg = 0x7f4c0288c880 tsep = 0x7f4c0a2800a0 #4 0x00000037d10079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00000037d0ce8b6d in ?? () No symbol table info available. #6 0x0000000000000000 in ?? () No symbol table info available. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-bugs@REDACTED Tue Sep 30 12:25:34 2014 From: raimo+erlang-bugs@REDACTED (Raimo Niskanen) Date: Tue, 30 Sep 2014 12:25:34 +0200 Subject: [erlang-bugs] bind_to is ignored in snmpm_net_if In-Reply-To: References: Message-ID: <20140930102534.GA29420@erix.ericsson.se> On Mon, Sep 29, 2014 at 01:29:54PM +0400, Pavel Baturko wrote: > Hi, > > I'm looking into erlang snmp manager code because my manager stops working > when I updated erlang from 17.1 to 17.3. > > The problem is that in otp/lib/snmp/src/manager/snmpm_net_if.erl (github > erlang/otp, branch maint) in function socket_params parameter BindTo is > ignored in case of Family==inet and init > :get_argument(snmp_fd)==error: > > socket_params(Domain, {IpAddr, IpPort}, BindTo, CommonSocketOpts) -> > ... > case Family of > inet -> > case init:get_argument(snmp_fd) of > {ok, [[FdStr]]} -> > ... > error -> > {IpPort, [{ip, IpAddr} | SocketOpts]} <<<<<<<< * > end; > _ -> > ... > end. > > *: here ip option is added regardless of BindTo argument. > > in other cases branches this option is utilized. > > When address option is not specified in manager.conf snmp manager uses > 127.0.0.1 as default and socket is binding to 127.0.0.1. After that all > gen_upd:send fails with einval. Yes that is a bug. Thank you for reporting it. You can try this patch: diff --git a/lib/snmp/src/manager/snmpm_net_if.erl b/lib/snmp/src/manager/snmpm_net_if.erl index cb72871..b4cc165 100644 --- a/lib/snmp/src/manager/snmpm_net_if.erl +++ b/lib/snmp/src/manager/snmpm_net_if.erl @@ -319,7 +319,7 @@ socket_open(IpPort, SocketOpts) -> Socket end. -socket_params(Domain, {IpAddr, IpPort}, BindTo, CommonSocketOpts) -> +socket_params(Domain, {IpAddr, IpPort} = Addr, BindTo, CommonSocketOpts) -> Family = snmp_conf:tdomain_to_family(Domain), SocketOpts = case Family of @@ -340,15 +340,18 @@ socket_params(Domain, {IpAddr, IpPort}, BindTo, CommonSocketOpts) -> {0, [{fd, Fd} | SocketOpts]} end; error -> - {IpPort, [{ip, IpAddr} | SocketOpts]} + socket_params(SocketOpts, Addr, BindTo) end; _ -> - case BindTo of - true -> - {IpPort, [{ip, IpAddr} | SocketOpts]}; - _ -> - {IpPort, SocketOpts} - end + socket_params(SocketOpts, Addr, BindTo) + end. +%% +socket_params(SocketOpts, {IpAddr, IpPort}, BindTo) -> + case BindTo of + true -> + {IpPort, [{ip, IpAddr} | SocketOpts]}; + _ -> + {IpPort, SocketOpts} end. common_socket_opts(Opts) -> Best Regards / Raimo Niskanen, Erlang/OTP > > Thanks, > Pavel > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From tony@REDACTED Tue Sep 30 14:39:10 2014 From: tony@REDACTED (Tony Rogvall) Date: Tue, 30 Sep 2014 14:39:10 +0200 Subject: [erlang-bugs] Erlang vm beam.smp crash In-Reply-To: <6a7d6a4b.104f6.148c0eeb097.Coremail.liu1985629@163.com> References: <6a7d6a4b.104f6.148c0eeb097.Coremail.liu1985629@163.com> Message-ID: <6B0DD9F1-3AC2-443D-ADA3-32324A45F2D0@rogvall.se> I also found this: (on unix/mac, on windows use some other program) > Port = open_port({spawn, "cat"}, []). > erlang:port_set_data(Port, {1,2,3}). true > erlang:port_get_data(Port). ... hmm my cpu skyrockets and the computer dies a bit :-) then beam.smp(41283,0xb06bb000) malloc: *** mach_vm_map(size=1799625657810944) failed (error code=3) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug beam.smp(41283,0xb06bb000) malloc: *** mach_vm_map(size=1799625657810944) failed (error code=3) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug beam.smp(41283,0xb06bb000) malloc: *** mach_vm_map(size=1799625656766464) failed (error code=3) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug beam.smp(41283,0xb06bb000) malloc: *** mach_vm_map(size=1799625656766464) failed (error code=3) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Crash dump was written to: erl_crash.dump eheap_alloc: Cannot allocate 1799625656762408 bytes of memory (of type "heap_frag"). /Tony On 29 sep 2014, at 12:22, ??? wrote: > I use http://www.erlang.org/download/otp_src_17.0.tar.gz to build the erlang. > > BIF_RETTYPE port_get_data_1(BIF_ALIST_1) > { > /* > * This is not a signal. See comment above. > */ > Eterm res; > erts_aint_t data; > Port* prt; > > prt = data_lookup_port(BIF_P, BIF_ARG_1); > if (!prt) > BIF_ERROR(BIF_P, BADARG); > > data = erts_smp_atomic_read_ddrb(&prt->data); > if (!data) > BIF_ERROR(BIF_P, BADARG); //I add the two lines to correct it. > > if ((data & 0x3) != 0) { > res = (Eterm) (UWord) data; > ASSERT(is_immed(res)); > } > else { > ErtsPortDataHeap *pdhp = (ErtsPortDataHeap *) data; > Eterm *hp = HAlloc(BIF_P, pdhp->hsize); > res = copy_struct(pdhp->data, pdhp->hsize, &hp, &MSO(BIF_P)); > } > > BIF_RET(res); > } > > > (gdb) bt full > #0 0x0000000000514524 in port_get_data_1 (A__p=0x7f4bc0d66488, BIF__ARGS=) at beam/erl_bif_port.c:591 > pdhp = 0x0 > hp = > data = 0 > #1 0x000000000054d517 in process_main () at beam/beam_emu.c:2787 > bf = 0x514490 > result = 1688368833101607 > init_done = 1 > c_p = 0x7f4bc0d66488 > reds_used = 178536832 > x0 = 1688368833101607 > reg = 0x7f4c0aa44180 > HTOP = 0x7f4bc036d350 > E = 0x7f4bc0370b18 > I = 0x7f4bfb5c7af8 > FCALLS = 1984 > tmp_arg1 = 139963324058344 > tmp_arg2 = 15 > tmp_big = {139964436718400, 5662828} > freg = 0x7f4c0aa461c0 > neg_o_reds = 0 > arith_func = 0 > opcodes = {0x54c14a, 0x54b78e, 0x54c06a, 0x54c0eb, 0x54c2f8, 0x54cee5, 0x54cb67, 0x54e173, 0x54ec5c, 0x54ca4d, 0x54ca43, 0x54ca23, 0x54908b, 0x54c5fe, 0x54d5c8, 0x54d605, 0x54d5f6, 0x54957d, 0x549451, 0x54d366, 0x54d26d, > 0x54d29b, 0x54d063, 0x54d092, 0x54d245, 0x54d223, 0x54d176, 0x54d4bb, 0x54ca52, 0x54caa1, 0x54ba31, 0x54ba07, 0x54ba26, 0x54bf45, 0x54bf66, 0x54ccfb, 0x54c949, 0x546667, 0x54ca9c, 0x54674e, 0x546771, 0x54667a, 0x5466ab, > 0x5466dc, 0x546715, 0x5464cd, 0x54eaa8, 0x54e14e, 0x54ea52, 0x54eb16, 0x546795, 0x5467b6, 0x5467d8, 0x5467f5, 0x546823, 0x546852, 0x546870, 0x54689f, 0x5468cf, 0x5468fc, 0x54692a, 0x546957, 0x54699e, 0x5469e6, 0x546a14, > 0x546a5c, 0x546aa5, 0x546ad3, 0x546b02, 0x546b30, 0x546b78, 0x546bc1, 0x546bf0, 0x546c39, 0x54d3bf, 0x54b2ce, 0x54b39e, 0x54b3c9, 0x54e468, 0x54b3bf, 0x54b5f8, 0x54e1d9, 0x54b046, 0x54b0ac, 0x54b65f, 0x54ce89, 0x54b47a, > 0x54d44b, 0x54e23b, 0x54b4bd, 0x5493b7, 0x54947d, 0x54ddf7, 0x54df22, 0x54e011, 0x54d8dd, 0x54d60f, 0x54d694, 0x54d965, 0x54d9db, 0x54da59, 0x54db67, 0x54d718, 0x54d348, 0x54d2c2, 0x54d340, 0x54b103, 0x54d34d, 0x54d1a0, > 0x54d8d6, 0x54d857, 0x54d8b7, 0x54d6ad, 0x54d57d, 0x54d5ba, 0x54d810, 0x54d849, 0x549430, 0x54bbbe, 0x54e2f0, 0x54d38c, 0x54e331, 0x54e345, 0x54bab9, 0x54e357, 0x549185, 0x54e3c9, 0x54e3e9, 0x5494f1, 0x549306, 0x549515, > 0x54d6a5, 0x54bbc9, 0x54953a, 0x54d0ee, 0x54d0b2, 0x54df04, 0x54ddd1, 0x54bc6c, 0x54dede, 0x54dd60, 0x54dc7d, 0x54dcef, 0x54af53, 0x54afcc, 0x54e7ca, 0x54e7ff, 0x54e294, 0x54e2be, 0x54b6ca, 0x54d355, 0x54d211, 0x54cf47, > 0x54cf9a, 0x54d007, 0x54d12e, 0x54ec20, 0x54b559, 0x54b507, 0x546539, 0x54656a, 0x5465a3, 0x5465e0, 0x54e0ea, 0x54e400, 0x546504, 0x5464e2, 0x54ea08, 0x54be99, 0x54c53b, 0x54c5d7, 0x54e774, 0x54bee0, 0x54bf28, 0x54bf36, > 0x5464cd, 0x54b222, 0x546d1f, 0x546d3e, 0x546d81, 0x546da0, 0x546dd2, 0x546e29, 0x546e49, 0x546e7c, 0x546d04, 0x546d5e, 0x546e05, 0x546ca2, 0x546cbd, 0x546ce0, 0x546c83, 0x54e83c, 0x54b1cc, 0x54b278, 0x54eb5b, 0x54ba46, > 0x54c683, 0x54c726, 0x54c7b4...} > temp_bits = 139964436760704 > pt_arity = 139963334550664 > start_time = 0 > start_time_i = 0x0 > EBS = 0x7f4c0288c898 > #2 0x00000000004a081b in sched_thread_func (vesdp=0x7f4c0288c880) at beam/erl_process.c:7665 > callbacks = {arg = 0x7f4c02882380, wakeup = 0x4a21b0 , prepare_wait = 0x49e370 , wait = 0x49f6f0 , finalize_wait = 0x49e350 } > esdp = 0x7f4c0288c880 > no = 2 > #3 0x00000000005df676 in thr_wrapper (vtwd=) at pthread/ethread.c:110 > result = > res = 0x7fffd82d4b90 > twd = > thr_func = 0x4a0700 > arg = 0x7f4c0288c880 > tsep = 0x7f4c0a2800a0 > #4 0x00000037d10079d1 in start_thread () from /lib64/libpthread.so.0 > No symbol table info available. > #5 0x00000037d0ce8b6d in ?? () > No symbol table info available. > #6 0x0000000000000000 in ?? () > No symbol table info available. > > > > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" From lenrel2014@REDACTED Tue Sep 30 14:39:07 2014 From: lenrel2014@REDACTED (Lenrel Lenrel) Date: Tue, 30 Sep 2014 14:39:07 +0200 Subject: [erlang-bugs] beam.smp hangs and use 100% of cpu In-Reply-To: References: Message-ID: I realised that the reason for this problem is related to KVM VM cpu model. The problem can be reproduced every time on standard qemu64 cpu model. After I change the model to "host", the problem is not happening any more. Maybe the there is some issue with qemu emulator. However, this is the configuration of CPU model: vendor_id : GenuineIntel cpu family : 6 model : 2 model name : QEMU Virtual CPU version 1.4.2 stepping : 3 microcode : 0x1 cpu MHz : 3200.080 cache size : 4096 KB physical id : 3 siblings : 1 core id : 0 cpu cores : 1 apicid : 3 initial apicid : 3 fpu : yes fpu_exception : yes cpuid level : 4 wp : yes flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni cx16 popcnt hypervisor lahf_lm bogomips : 6400.16 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: Cmdline arguments: /usr/lib/erlang/erts-6.2/bin/beam.smp-Ww-Ktrue-A30-P1048576---root/usr/lib/erlang-prognameerl---home/root---pa./sbin/../ebin-noshell-noinput-srabbitboot-snamerabbit@REDACTED [{nodelay,true}]-saslerrlog_typeerror-saslsasl_error_loggerfalse-rabbiterror_logger{file,"./sbin/../var/log/rabbitmq/rabbit@REDACTED "}-rabbitsasl_error_logger{file,"./sbin/../var/log/rabbitmq/rabbit@REDACTED "}-rabbitenabled_plugins_file"./sbin/../etc/rabbitmq/enabled_plugins"-rabbitplugins_dir"./sbin/../plugins"-rabbitplugins_expand_dir"./sbin/../var/lib/rabbitmq/mnesia/rabbit@REDACTED "-os_monstart_cpu_supfalse-os_monstart_disksupfalse-os_monstart_memsupfalse-mnesiadir"./sbin/../var/lib/rabbitmq/mnesia/rabbit@REDACTED "-kernelinet_dist_listen_min25672-kernelinet_dist_listen_max25672 It doesn't crash, just hangs. After sending ABRT signal to beam.smp COREDUMP: https://www.dropbox.com/s/8sirk9owet8c5w0/coredump.xz?dl=0 COMPILE options: ./configure \ --prefix=/usr \ --enable-smp-support \ --with-odbc I'm using Snowshoe Linux S14 / 64 bit. Regards, On Sat, Sep 27, 2014 at 3:29 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Fri, Sep 26, 2014 at 4:26 PM, Lenrel Lenrel > wrote: > >> I have compiled from source erlang 17.3 and rabbitmq 3.3.5. > > > We need more information: > > System architecture / operating system > What compilation options were used > Do you use HiPE? > arguments to the VM > > Can you log in to the console on the node? > Do you have a crashdump? (I believe sending SIGQUIT to the beam.smp > process will produce one, but do note environment variables in the erl(1) > man page). > > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: