[erlang-questions] base64:decode fails on facebook base64 encoded strings

[Michael Santos]

On Thu, Dec 29, 2011 at 08:30:09AM +0400, Max Bourinov wrote:
> So Erlangers,
> 
> I wrote about this little bug in Facebook and I got an answer!
> 
> >>I believe the trailing = is an optional padding in base64 your code needs
> to accept base64 with and without trailing '=' characters. See last part of
> http://email.about.com/cs/standards/a/base64_encoding.htm
> 
> This is what they said. That is very interesting, I personally would not
> omit data from protocol specification, but it appears that if it possible.
> Anyway I didn't see anything about omitting here
> http://tools.ietf.org/html/rfc1421 or http://tools.ietf.org/html/rfc2045

There is something about dropping the padding in RFC 4648:

5.  Base 64 Encoding with URL and Filename Safe Alphabet

The pad character "=" is typically percent-encoded when used in an URI
[9], but if the data length is known implicitly, this can be avoided by
skipping the padding; see section 3.2.

> So, if you ever face such a loose implementation of base64 here is a
> snippet that might help you to deal with it:
> 
> -spec fb_decode_base64(Base64 :: list()) -> binary() | error.
> fb_decode_base64(Base64) when is_list(Base64) ->
>     try base64:decode(Base64)
>     catch
>         error:_ -> % could be missing =
>         	try base64:decode(Base64 ++ "=")
>         	catch
>         		error:_ -> % could be missing ==
>         			try base64:decode(Base64 ++ "==")
>         			catch
>         				error:_ -> % base64 is really wrong. we cannot fix it
>         					error
>         			end
>         	end
> 
>     end.

The length can be used to figure out the padding (not exactly the same
as your code, throws an exception on bad data rather than returning
an error):

-module(tt).
-export([fb_decode_base64/1]).

fb_decode_base64(Base64) when is_list(Base64) ->
    decode(list_to_binary(Base64)).

decode(Base64) when byte_size(Base64) rem 4 == 3 ->
    base64:decode(<<Base64/bytes, "=">>);
decode(Base64) when byte_size(Base64) rem 4 == 2 ->
    base64:decode(<<Base64/bytes, "==">>);
decode(Base64) ->
    base64:decode(Base64).