[erlang-bugs] possible bug in ssl and/or public_key module (R13 and R14)
Filipe David Manana
fdmanana@REDACTED
Wed Sep 22 17:11:44 CEST 2010
Ingela, some progress with that branch:
2> ssl_test:test().
=INFO REPORT==== 22-Sep-2010::16:10:13 ===
SSL WARNING: Ignoring CA cert:
<<48,130,3,251,48,130,2,227,160,3,2,1,2,2,1,1,
48,13,6,9,42,134,72,134,247,13,1,1,5,5,0,48,
129,183,49,63,48,61,6,3,85,4,3,12,54,84,195,
156,82,75,84,82,85,83,84,32,69,108,101,107,
116,114,111,110,105,107,32,83,101,114,116,105,
102,105,107,97,32,72,105,122,109,101,116,32,
83,97,196,159,108,97,121,196,177,99,196,177,
115,196,177,49,11,48,9,6,3,85,4,6,12,2,84,82,
49,15,48,13,6,3,85,4,7,12,6,65,78,75,65,82,65,
49,86,48,84,6,3,85,4,10,12,77,40,99,41,32,50,
48,48,53,32,84,195,156,82,75,84,82,85,83,84,
32,66,105,108,103,105,32,196,176,108,101,116,
105,197,159,105,109,32,118,101,32,66,105,108,
105,197,159,105,109,32,71,195,188,118,101,110,
108,105,196,159,105,32,72,105,122,109,101,116,
108,101,114,105,32,65,46,197,158,46,48,30,23,
13,48,53,48,53,49,51,49,48,50,55,49,55,90,23,
13,49,53,48,51,50,50,49,48,50,55,49,55,90,48,
129,183,49,63,48,61,6,3,85,4,3,12,54,84,195,
156,82,75,84,82,85,83,84,32,69,108,101,107,
116,114,111,110,105,107,32,83,101,114,116,105,
102,105,107,97,32,72,105,122,109,101,116,32,
83,97,196,159,108,97,121,196,177,99,196,177,
115,196,177,49,11,48,9,6,3,85,4,6,12,2,84,82,
49,15,48,13,6,3,85,4,7,12,6,65,78,75,65,82,65,
49,86,48,84,6,3,85,4,10,12,77,40,99,41,32,50,
48,48,53,32,84,195,156,82,75,84,82,85,83,84,
32,66,105,108,103,105,32,196,176,108,101,116,
105,197,159,105,109,32,118,101,32,66,105,108,
105,197,159,105,109,32,71,195,188,118,101,110,
108,105,196,159,105,32,72,105,122,109,101,116,
108,101,114,105,32,65,46,197,158,46,48,130,1,
34,48,13,6,9,42,134,72,134,247,13,1,1,1,5,0,3,
130,1,15,0,48,130,1,10,2,130,1,1,0,202,82,5,
214,99,3,216,28,95,221,210,123,93,242,12,96,
97,91,107,59,116,43,120,13,125,69,189,34,116,
232,140,3,193,198,17,42,61,149,188,169,148,
176,187,145,151,200,105,124,132,197,180,145,
108,108,19,106,164,85,173,164,133,232,149,126,
179,0,175,0,194,5,24,245,112,157,54,139,174,
203,228,27,129,127,147,136,251,106,85,187,125,
133,146,206,186,88,159,219,50,197,189,93,239,
34,74,47,65,7,126,73,97,179,134,236,78,166,65,
110,132,188,3,236,245,59,28,200,31,194,238,
168,238,234,18,74,141,20,207,243,10,224,80,57,
249,8,53,248,17,89,173,231,34,234,75,202,20,6,
222,66,186,178,153,243,45,84,136,16,6,234,225,
26,62,61,103,31,251,206,251,124,130,232,17,93,
74,193,185,20,234,84,217,102,155,124,137,125,
4,154,98,201,233,82,60,158,156,239,210,245,38,
228,230,229,24,124,139,110,223,108,204,120,91,
79,114,178,203,92,63,140,5,141,209,76,140,173,
146,199,225,120,127,101,108,73,6,80,44,158,50,
194,215,74,198,117,138,89,78,117,111,71,94,
193,2,3,1,0,1,163,16,48,14,48,12,6,3,85,29,19,
4,5,48,3,1,1,255,48,13,6,9,42,134,72,134,247,
13,1,1,5,5,0,3,130,1,1,0,21,245,85,255,55,150,
128,89,33,164,252,161,21,76,32,246,212,95,218,
3,36,252,207,144,26,244,33,10,154,238,58,177,
106,239,239,248,96,209,76,54,102,69,29,243,
102,2,116,4,123,146,48,168,222,10,118,15,239,
149,110,189,201,55,230,26,13,172,137,72,91,
204,131,54,194,245,70,92,89,130,86,180,213,
254,35,180,216,84,28,68,171,196,167,229,20,
206,60,65,97,124,67,230,205,196,129,9,139,36,
251,84,37,214,22,168,150,12,103,7,111,179,80,
71,227,28,36,40,221,42,152,164,97,254,219,234,
18,55,188,1,26,52,133,189,110,79,231,145,114,
7,68,133,30,88,202,84,68,221,247,172,185,203,
137,33,114,219,143,192,105,41,151,42,163,174,
24,35,151,28,65,42,139,124,42,193,124,144,232,
169,40,192,211,145,198,173,40,135,64,104,181,
255,236,167,210,211,56,24,156,211,125,105,93,
240,198,165,30,36,27,163,71,252,105,7,104,231,
228,154,180,237,15,161,135,135,2,206,135,210,
72,78,225,188,255,203,241,114,146,68,100,3,37,
234,222,91,110,159,201,242,78,172,221,199>>
Due to decoding error:{badmatch,
{error,
{asn1,
{{case_clause,19},
[{'OTP-PUB-KEY',
check_and_convert_restricted_string,5},
{'OTP-PUB-KEY',decode,2},
{pubkey_cert_records,transform,2},
{lists,map,2},
{lists,map,2},
{pubkey_cert_records,transform,2},
{pubkey_cert_records,decode_tbs,1},
{pubkey_cert_records,decode_cert,1}]}}}}
Reason: {bad_cert,missing_basic_constraint}
=ERROR REPORT==== 22-Sep-2010::16:10:13 ===
SSL: certify_certificate: ./ssl_handshake.erl:586:Fatal error: handshake
failure
** exception error: no match of right hand side value {error,esslconnect}
in function ssl_test:test/0
3>
Does this error rings a bell?
cheers
On Wed, Sep 22, 2010 at 2:11 PM, Ingela Anderton Andin <
ingela@REDACTED> wrote:
> Hi again, something went wrong with the branch please look at the
> branch ia/ssl-and-public_key/backwards-compatibility/OTP-8858 instead.
>
>
> Regards Ingela Erlang/OTP team - Ericsson AB
>
> Ingela Anderton Andin wrote:
>
>> Hi again!
>>
>> Well ok, first I would like you to provide the option verify_fun:
>>
>> FunAndState = {fun(_,{bad_cert, _} = Reason, _) ->
>> io:format("Reason: ~p~n", [Reason]),
>> {fail, Reason};
>> (_,{extension, _}, UserState) ->
>> {unknown, UserState};
>> (_, valid, UserState) ->
>> {valid, UserState}
>> end, []},
>>
>> add option:
>> {verify_fun, FunAndState}
>>
>> So we can try to find out why it does not like the cert.
>>
>> By the way we decided to shorten the INFO report if you would like to run
>> the latest ssl it is now
>> on the branch ia/ssl/public_key/backwards-compatibility/OTP-8858.
>>
>>
>
>
>
> Regards Ingela Erlang/OTP team - Ericsson AB
>>
>> Filipe David Manana wrote:
>>
>>> On Tue, Sep 21, 2010 at 4:07 PM, Ingela Anderton Andin <
>>> ingela@REDACTED <mailto:ingela@REDACTED>> wrote:
>>>
>>> Hi!
>>>
>>> Yes you could, it is a INFO report a warning that that particular
>>> CA cert is ignored as we could not decode it.
>>> But you get another handshake error. What options do you connect
>>> with ?
>>>
>>>
>>> This is an excerpt of my testing code:
>>>
>>> Options = [
>>> {ssl_imp, new},
>>> binary,
>>> {nodelay, true},
>>> {active, false},
>>> {verify, verify_peer},
>>> {depth, 3},
>>> {cacertfile, "/etc/ssl/certs/ca-certificates.crt"}
>>> ],
>>> {ok, S} = ssl:connect(?HOST, 443, Options),
>>> ok = ssl:send(S, Body),
>>> loop(S),
>>> ssl:close(S).
>>>
>>> loop(S) ->
>>> ssl:setopts(S, [{active, once}]),
>>> receive
>>> {ssl, S, Data} ->
>>> io:format("received data: ~p~n", [Data]),
>>> loop(S);
>>> {ssl_closed, S} ->
>>> io:format("socket closed", []);
>>> {ssl_error, S, Error} ->
>>> io:format("socket error: ~p~n", [Error])
>>> end.
>>>
>>> Once again, thanks for looking into this.
>>>
>>>
>>>
>>> Regards Ingela Erlang/OTP team - Ericsson AB
>>>
>>>
>>> Filipe David Manana wrote:
>>>
>>> Ingela,
>>>
>>> After pulling your last commit, things advance a bit more, but
>>> still not able to open the CAs file:
>>>
>>> =INFO REPORT==== 21-Sep-2010::14:35:58 ===
>>> SSL WARNING: Ignoring CA cert:
>>> <<48,130,3,251,48,130,2,227,160,3,2,1,2,2,1,1,
>>>
>>> 48,13,6,9,42,134,72,134,247,13,1,1,5,5,0,48,
>>>
>>> 129,183,49,63,48,61,6,3,85,4,3,12,54,84,195,
>>>
>>> 156,82,75,84,82,85,83,84,32,69,108,101,107,
>>>
>>> 116,114,111,110,105,107,32,83,101,114,116,105,
>>>
>>> 102,105,107,97,32,72,105,122,109,101,116,32,
>>>
>>> 83,97,196,159,108,97,121,196,177,99,196,177,
>>>
>>> 115,196,177,49,11,48,9,6,3,85,4,6,12,2,84,82,
>>>
>>> 49,15,48,13,6,3,85,4,7,12,6,65,78,75,65,82,65,
>>>
>>> 49,86,48,84,6,3,85,4,10,12,77,40,99,41,32,50,
>>>
>>> 48,48,53,32,84,195,156,82,75,84,82,85,83,84,
>>>
>>> 32,66,105,108,103,105,32,196,176,108,101,116,
>>>
>>> 105,197,159,105,109,32,118,101,32,66,105,108,
>>>
>>> 105,197,159,105,109,32,71,195,188,118,101,110,
>>>
>>> 108,105,196,159,105,32,72,105,122,109,101,116,
>>>
>>> 108,101,114,105,32,65,46,197,158,46,48,30,23,
>>>
>>> 13,48,53,48,53,49,51,49,48,50,55,49,55,90,23,
>>>
>>> 13,49,53,48,51,50,50,49,48,50,55,49,55,90,48,
>>>
>>> 129,183,49,63,48,61,6,3,85,4,3,12,54,84,195,
>>>
>>> 156,82,75,84,82,85,83,84,32,69,108,101,107,
>>>
>>> 116,114,111,110,105,107,32,83,101,114,116,105,
>>>
>>> 102,105,107,97,32,72,105,122,109,101,116,32,
>>>
>>> 83,97,196,159,108,97,121,196,177,99,196,177,
>>>
>>> 115,196,177,49,11,48,9,6,3,85,4,6,12,2,84,82,
>>>
>>> 49,15,48,13,6,3,85,4,7,12,6,65,78,75,65,82,65,
>>>
>>> 49,86,48,84,6,3,85,4,10,12,77,40,99,41,32,50,
>>>
>>> 48,48,53,32,84,195,156,82,75,84,82,85,83,84,
>>>
>>> 32,66,105,108,103,105,32,196,176,108,101,116,
>>>
>>> 105,197,159,105,109,32,118,101,32,66,105,108,
>>>
>>> 105,197,159,105,109,32,71,195,188,118,101,110,
>>>
>>> 108,105,196,159,105,32,72,105,122,109,101,116,
>>>
>>> 108,101,114,105,32,65,46,197,158,46,48,130,1,
>>>
>>> 34,48,13,6,9,42,134,72,134,247,13,1,1,1,5,0,3,
>>>
>>> 130,1,15,0,48,130,1,10,2,130,1,1,0,202,82,5,
>>>
>>> 214,99,3,216,28,95,221,210,123,93,242,12,96,
>>>
>>> 97,91,107,59,116,43,120,13,125,69,189,34,116,
>>>
>>> 232,140,3,193,198,17,42,61,149,188,169,148,
>>>
>>> 176,187,145,151,200,105,124,132,197,180,145,
>>>
>>> 108,108,19,106,164,85,173,164,133,232,149,126,
>>>
>>> 179,0,175,0,194,5,24,245,112,157,54,139,174,
>>>
>>> 203,228,27,129,127,147,136,251,106,85,187,125,
>>>
>>> 133,146,206,186,88,159,219,50,197,189,93,239,
>>>
>>> 34,74,47,65,7,126,73,97,179,134,236,78,166,65,
>>>
>>> 110,132,188,3,236,245,59,28,200,31,194,238,
>>>
>>> 168,238,234,18,74,141,20,207,243,10,224,80,57,
>>>
>>> 249,8,53,248,17,89,173,231,34,234,75,202,20,6,
>>>
>>> 222,66,186,178,153,243,45,84,136,16,6,234,225,
>>>
>>> 26,62,61,103,31,251,206,251,124,130,232,17,93,
>>>
>>> 74,193,185,20,234,84,217,102,155,124,137,125,
>>>
>>> 4,154,98,201,233,82,60,158,156,239,210,245,38,
>>>
>>> 228,230,229,24,124,139,110,223,108,204,120,91,
>>>
>>> 79,114,178,203,92,63,140,5,141,209,76,140,173,
>>>
>>> 146,199,225,120,127,101,108,73,6,80,44,158,50,
>>>
>>> 194,215,74,198,117,138,89,78,117,111,71,94,
>>>
>>> 193,2,3,1,0,1,163,16,48,14,48,12,6,3,85,29,19,
>>>
>>> 4,5,48,3,1,1,255,48,13,6,9,42,134,72,134,247,
>>>
>>> 13,1,1,5,5,0,3,130,1,1,0,21,245,85,255,55,150,
>>>
>>> 128,89,33,164,252,161,21,76,32,246,212,95,218,
>>>
>>> 3,36,252,207,144,26,244,33,10,154,238,58,177,
>>>
>>> 106,239,239,248,96,209,76,54,102,69,29,243,
>>>
>>> 102,2,116,4,123,146,48,168,222,10,118,15,239,
>>>
>>> 149,110,189,201,55,230,26,13,172,137,72,91,
>>>
>>> 204,131,54,194,245,70,92,89,130,86,180,213,
>>>
>>> 254,35,180,216,84,28,68,171,196,167,229,20,
>>>
>>> 206,60,65,97,124,67,230,205,196,129,9,139,36,
>>>
>>> 251,84,37,214,22,168,150,12,103,7,111,179,80,
>>>
>>> 71,227,28,36,40,221,42,152,164,97,254,219,234,
>>>
>>> 18,55,188,1,26,52,133,189,110,79,231,145,114,
>>>
>>> 7,68,133,30,88,202,84,68,221,247,172,185,203,
>>>
>>> 137,33,114,219,143,192,105,41,151,42,163,174,
>>>
>>> 24,35,151,28,65,42,139,124,42,193,124,144,232,
>>>
>>> 169,40,192,211,145,198,173,40,135,64,104,181,
>>>
>>> 255,236,167,210,211,56,24,156,211,125,105,93,
>>>
>>> 240,198,165,30,36,27,163,71,252,105,7,104,231,
>>>
>>> 228,154,180,237,15,161,135,135,2,206,135,210,
>>>
>>> 72,78,225,188,255,203,241,114,146,68,100,3,37,
>>>
>>> 234,222,91,110,159,201,242,78,172,221,199>>
>>> Due to decoding error:{badmatch,
>>> {error,
>>> {asn1,
>>> {{case_clause,19},
>>> [{'OTP-PUB-KEY',
>>>
>>> check_and_convert_restricted_string,5},
>>> {'OTP-PUB-KEY',decode,2},
>>> {pubkey_cert_records,transform,2},
>>> {lists,map,2},
>>> {lists,map,2},
>>> {pubkey_cert_records,transform,2},
>>> {pubkey_cert_records,decode_tbs,1},
>>>
>>> {pubkey_cert_records,decode_cert,1}]}}}}
>>>
>>>
>>> =ERROR REPORT==== 21-Sep-2010::14:35:58 ===
>>> SSL: certify_certificate: ./ssl_handshake.erl:584:Fatal error:
>>> handshake failure
>>> ** exception error: no match of right hand side value
>>> {error,esslconnect}
>>> in function ssl_test:test/0
>>>
>>>
>>> cheers
>>>
>>> On Tue, Sep 21, 2010 at 1:07 PM, Ingela Anderton Andin
>>> <ingela@REDACTED <mailto:ingela@REDACTED>
>>> <mailto:ingela@REDACTED
>>> <mailto:ingela@REDACTED>>> wrote:
>>>
>>> Hi!
>>>
>>> Filipe David Manana wrote:
>>>
>>> On Mon, Sep 20, 2010 at 2:47 PM, Ingela Anderton Andin <
>>> ingela@REDACTED
>>> <mailto:ingela@REDACTED>
>>> <mailto:ingela@REDACTED
>>> <mailto:ingela@REDACTED>>> wrote:
>>>
>>> So I definitely consider this a
>>> regression :(
>>> The weird thing is that I can use this certificates
>>> file with the old ssl
>>> implementation (default on R13 and R12 releases) on
>>> R13B03
>>> and R13B04 at
>>> least.
>>> Well the thing is that the old ssl-implementation
>>> only is
>>> an erlang-glue
>>> that leaves the most things up to the underlaying
>>> openssl
>>> implementation,
>>> but the new ssl
>>> only uses openssl crypto library and takes care the ssl
>>> protocol
>>> fsm-machinery and certificate handling on its
>>> own. This
>>> makes many things
>>> much easier to implement
>>> and removes the bottleneck enforced by the glue,
>>> and also
>>> lessens the
>>> required resource allocation. Of course this may
>>> cause new
>>> bugs occasionally
>>> and we fix them
>>> as fast as we can.
>>> If you want to try out the latest changes to fix the
>>> DSS-Params bug you can
>>> get the branch ia/ssl-asn1-spec-dss-params at my github
>>> account
>>> git@REDACTED:IngelaAndin/otp.git
>>>
>>> Hi,
>>>
>>> Ingela, I tried your git branch
>>> ssl-asn1-spec-dss-params but
>>> unfortunatelly
>>> I still get an exception:
>>>
>>> =ERROR REPORT==== 21-Sep-2010::11:57:03 ===
>>> SSL: 1060: error:{error,
>>> {asn1,
>>> {{case_clause,19},
>>> [{'OTP-PUB-KEY',
>>>
>>> check_and_convert_restricted_string,5},
>>> {'OTP-PUB-KEY',decode,2},
>>>
>>> {pubkey_cert_records,transform,2},
>>> {lists,map,2},
>>> {lists,map,2},
>>>
>>> {pubkey_cert_records,transform,2},
>>>
>>> {pubkey_cert_records,decode_tbs,1},
>>>
>>> {pubkey_cert_records,decode_cert,2}]}}}
>>> /home/fdmanana/tmp/ibrowse-test/ca-certificates.crt
>>> []
>>>
>>> ** exception error: no match of right hand side value
>>> {error,ecacertfile}
>>>
>>>
>>> Yes I get this too, it seems to be that one of
>>> the
>>> certificates in
>>> the file
>>> has a field that is utf8-encoded but the asn-1-spec says
>>> that it
>>> should
>>> be a "printableString". I do not know if openssl tries to
>>> decode
>>> it, it might
>>> not until it is used, and it might not be. Erlang ssl
>>> caches all
>>> cert in the ca-file.
>>> I have now made new ssl more tolerant so that it
>>> will ignore such ca-certs, that does not follow the spec.
>>> I have
>>> pushed the change to
>>> the ia/ssl-asn1-spec-dss-params branch.
>>>
>>>
>>> And btw, with the old ssl implementation, using a ssl
>>> socket
>>> in {active,
>>> once} mode, I receive very often an error like this:
>>>
>>> [Thu, 16 Sep 2010 00:10:34 GMT] [error] [<0.604.0>] **
>>> Generic
>>> server
>>> <0.604.0> terminating
>>> ** Last message in was {tcp,#Port<0.2288>,
>>>
>>>
>>> <<"\r\n6d\r\n,\n{\"seq\":70,\"id\":\"97b36d5003934d0c9dd58057b05fa167\",\"changes\":[{\"rev\":\"1-0d6deda5b380ae207ba87a7a3a32d0a1\"}]}\r\n6d\r\n,\n{\"seq\":71,\"id\":\"8a1c475b8dc5426e9172d6b970ae7c03\",\"changes\":[{\"rev\":\"1-72851f645fb6ab77f36866cbe505d82c\"}]}\r\n6d\r\n,\n{\"seq\":72,\"id\":\"fdb1d5b1c5b24ce481463ad668c13c40\",\"changes\":[{\"rev\":\"1-c37b5444eec8375631c326a0e77ca427\"}]}\r\n6d\r\n,\n{\"seq\":73,\"id\":\"b612465dafc44699b09d8bef5d4d4d8d\",\"changes\":[{\"rev\":\"1-be951f78ba830f5a1002abe0ce479c2d\"}]}\r\n6d\r\n,\n{\"seq\":74,\"id\":\"d2c2b5a771ef4b57b6d58fce2808cf7c\",\"changes\":[{\"rev\":\"1-c628443ff4dd7c3d9b4fd226727e2841\"}]}\r\n6d\r\n,\n{\"seq\":75,\"id\":\"8d669c377f08442981ce2d18a21d920b\",\"changes\":[{\"rev\":\"1-6db3a14c76701b87b0686412093ac103\"}]}\r\n6d\r\n,\n{\"seq\":76,\"id\":\"367bf0948d9d459582d187c9232844b8\",\"changes\":[{\"rev\":\"1-16ae7cf1c04c4f7c024493de1f18c8ed\"}]}\r\n6d\r\n,\n{\"seq\":77,\"id\":\"f2c805327ae740098e5db221c3f27b4b\",\"changes\":[{\"rev\":\"1-b22aa541f7e353a4cd430a9293239c77\"}]}\r\n6d\r\n,\n{\"seq\":78,\"id\":\"6ddf8033cec845c8986ee4bd03ff8ed6\",\"changes\":[{\"rev\":\"1-23f5957d250f5079277e6e4a86def1f1\"}]}\r\n6d\r\n,\n{\"seq\":79,\"id\":\"738365bd4fed44158516211847c13616\",\"changes\":[{\"rev\":\"1-6dcd375366f107fb2575c8eda6c6bdec\"}]}\r\n6d\r\n,\n{\"seq\":80,\"id\":\"2d66c797761b4506934d00b2fd260f90\",\"changes\":[{\"rev\":\"1-cc7dddd31fd753a9b4577607ce321cef\"}]}\r\n6d\r\n,\n{\"seq\":81,\"id\":\"0c01c012d4f540a3a015d57681a0af4f\",\"changes\":[{\"rev\":\"1-ff288fbba546fbfbf78c602e2fa39ea2\"}]}\r\n6d\r\n,\n{\"seq\":82,\"id\":\"dc8a7ff04d37428ea83c3515a801bd32\",\"changes\":[{\"rev\":\"1-2">>}
>>>
>>> ** When Server state ==
>>> {st,connector,<0.119.0>,<0.603.0>,<0.603.0>,11,false,
>>> [{mode,binary},
>>> {nodelay,true},
>>> {active,once},
>>> {packet,0},
>>> {ip,{0,0,0,0}},
>>> {verify,0},
>>> {depth,1}],
>>> {sslsocket,11,<0.604.0>},
>>>
>>> #Port<0.2288>,nil,open,false,false}
>>>
>>>
>>> The data, third argument of the tuple, is what is
>>> supposed to
>>> be. However
>>> the ssl module trows that exception (since it was
>>> expecting to
>>> receive only
>>> messages like {ssl, Socket, Data}). Is this a known issue?
>>>
>>>
>>> Humm ... not that I know of. We are aiming to
>>> remove the old
>>> ssl-implementation as soon as the new one is compleate
>>> enough and
>>> in first hand we do not fix things in the old implementation.
>>>
>>>
>>> Regards Ingela Erlang/OTP team - Ericsson AB
>>>
>>>
>>>
>>>
>>>
>>>
>>> ________________________________________________________________
>>> erlang-bugs (at) erlang.org <http://erlang.org>
>>> <http://erlang.org> mailing list.
>>>
>>> See http://www.erlang.org/faq.html
>>> To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED
>>> <mailto:erlang-bugs-unsubscribe@REDACTED>
>>> <mailto:erlang-bugs-unsubscribe@REDACTED
>>> <mailto:erlang-bugs-unsubscribe@REDACTED>>
>>>
>>>
>>>
>>>
>>> -- Filipe David Manana,
>>> fdmanana@REDACTED <mailto:fdmanana@REDACTED>
>>> <mailto:fdmanana@REDACTED <mailto:fdmanana@REDACTED>>,
>>> fdmanana@REDACTED <mailto:fdmanana@REDACTED>
>>> <mailto:fdmanana@REDACTED <mailto:fdmanana@REDACTED>>
>>>
>>>
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>>>
>>>
>>>
>>>
>>> ________________________________________________________________
>>> erlang-bugs (at) erlang.org <http://erlang.org> mailing list.
>>> See http://www.erlang.org/faq.html
>>> To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED
>>> <mailto:erlang-bugs-unsubscribe@REDACTED>
>>>
>>>
>>>
>>>
>>> --
>>> Filipe David Manana,
>>> fdmanana@REDACTED <mailto:fdmanana@REDACTED>, fdmanana@REDACTED<mailto:
>>> fdmanana@REDACTED>
>>>
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>>>
>>>
>>
>>
>> ________________________________________________________________
>> erlang-bugs (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-bugs-unsubscribe@REDACTED
>>
>>
>>
>
>
--
Filipe David Manana,
fdmanana@REDACTED, fdmanana@REDACTED
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
More information about the erlang-bugs
mailing list