[erlang-questions] Performance of base64 decode vs mime_decode

[Jesse Stimpson]

Hello,

While profiling my app I came across a significant difference in the
performance between base64:decode/1 and base64:mime_decode/1. The
implementations are very similar, but based on my experimentation, I think
the impl of mime_decode is preventing the Erlang compiler from enabling
some tail recursion optimizations.

Unfortunately, I don't have deep enough knowledge of the compiler to
confirm this directly, but I did make some modifications and was able to
achieve some better results.

Here are my findings:

27> base64_timing:run().
Function                  |  Time     |  Rel
--------------------------+-----------+-----
base64:decode/1             287838 us   1.0
base64:mime_decode/1        493222 us   1.71
base64_edits:mime_decode/1  270877 us   0.94
    TESTS ok
ok

Note: The timing test provided in base64_timing.erl creates a 7 MB
base64-encoded binary and decodes it using the various functions. It also
includes a copy of the unit tests for base64:mime_decode/1 that appear in
the Erlang 19.1 release.

The basic idea of the modifications in base64_edits is a focus on
maintaining clean tail recursion throughout the implementation. I found
that any expression that would "collapse" the tail (T) by reading it in
full (specifically: pattern matching against <<>>) would nearly double the
execution time. The decode map is simply copied from base64.erl for
convenience.

With that, my questions --
1. Is there a flaw in my measurements and/or mime_decode modifications?
2. Is there interest in me submitting a patch with these findings?

Thanks!
Jesse Stimpson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20161129/831ee6fc/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: base64_timing.erl
Type: application/octet-stream
Size: 2634 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20161129/831ee6fc/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: base64_edits.erl
Type: application/octet-stream
Size: 3052 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20161129/831ee6fc/attachment-0001.obj>