R13B03 makes SSL handshake fail (bug in OTP-PUB-KEY.erl).

[aude quintana]

Hello everybody,
Since the R13B03 release I have a problem with OTP-PUB-KEY.erl : SSL
handshake makes it bug.
It doens't accept certificates and other pem file any more.
The error is a case clause bug in decode/2.

I found the bug, the decode/2 function is like this :
decode(Type, Data) ->
   case catch decode(Data) of
     {error, incomplete} -> {error, incomplete};
     {Data, _Rest} ->
         case catch decode_disp(Type, Data) of
           {error, incomplete} -> {error, incomplete};
           {'EXIT', {error, Reason}} -> {error, Reason};
           {'EXIT', Reason} -> {error, {asn1, Reason}};
           Result -> {ok, Result}
         end
   end.

And decode(Data) when Data is a binary return something different from Data.
So the {Data, _Rest} never match whereas the result of decode(Data) is right,
so this is the case clause error.

I patched this file (replacing Data with Data0 in function argument
and in decode argument), and the problem disappeared.
I joined the patch but here is the corrected decode function (it's the
only change from the patch). :
decode(Type, Data0) ->
   case catch decode(Data0) of
     {error, incomplete} -> {error, incomplete};
     {Data, _Rest} ->
         case catch decode_disp(Type, Data) of
           {error, incomplete} -> {error, incomplete};
           {'EXIT', {error, Reason}} -> {error, Reason};
           {'EXIT', Reason} -> {error, {asn1, Reason}};
           Result -> {ok, Result}
         end
   end.

But this patch is not enough : OTP-PUB-KEY.erl is a generated file. I
found the gen file (probably asn1ct_gen.erl and
asn1ct_gen_ber_bin_v2.erl). The problem can be solved if
OTP-PUB-KEY.asn1 is generated with undec_rest option. I tried to add
this option in the Makefile
(otp_src_R13B03/lib/public_key/asn1/Makefile), it had a right effect
on decode/2 but it added some bugs on others functions.
So now, I have no time to properly fix this bug. I continue with my
patch and I hope someone will find the way to fix the bug from the
asn1 gen files.

Best Regards,
--
Aude Quintana
Mobile Devices
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OTP-PUB-KEY.diff
Type: text/x-patch
Size: 608 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20091223/b8cada1f/attachment.bin>