[erlang-patches] Optimization of beams string table generation

[Björn Gustavsson]

On Sun, Jul 4, 2010 at 3:19 PM, Paul Guyot <pguyot@REDACTED> wrote:
> Hello,
>
> beam_dict:string and the whole string table code is based on naive string search and compiling a module with thousands of binary strings takes several minutes. The following patch replaces the code with calls to binary:match/2.
>
> git fetch git://github.com/pguyot/otp.git pg/optimize_beam_dict_string_table
>
> http://github.com/pguyot/otp/commit/13ca6aa7b74c7887ae6c3a42b35769524af4903a
>

Can you give some more information about the claims in the commit
message?

I did the following in $ERL_TOP/release/tests/stdlib_test
(using the R14A compiler):

erlc +time -I ../test_server re_testoutput1_split_test.erl

The interesting part of the output are the following lines:

 core_module                   :      10.26 s   60987.5 kB
 v3_codegen                    :       5.83 s   17908.4 kB
 beam_asm                      :       0.46 s       2.5 kB

The module contains several thousand strings
that are used in bit syntax matching. The core_module
and v3_codegen passes are very slow. The beam_asm
pass is slower than usual, but fast compared to the
slow passes.

Do you have a module that will make the beam_asm
pass very slow?

If not, the change is still interesting because the new
code is shorter and easier to understand, but the
commit message must be revised.

Regarding the implementation, it will probably be
faster to append new strings to the string table
like this:

	    NewDict = Dict#asm{strings = <<Strings/binary,StrBin/binary>>,
			       string_offset=NextOffset+byte_size(StrBin)},

Regarding the test case, a better place for it should be the
compilation_SUITE module.

(The name of the test modules may not be the best, but
compile_SUITE (mainly) tests different compiler options,
while compilation_SUITE tests miscellanous language features.)


-- 
Björn Gustavsson, Erlang/OTP, Ericsson AB