[erlang-questions] Small performance hint: avoid abusing module_info(attributes)

[Ulf Wiger]

At the very least, one should exercise caution on some particular modules,
and perhaps generally modules that have lots of type specs.

Interestingly, when fetching attributes from beam_lib, dialyzer-generated
attributes are moved to a subgroup, so at least in OTP 20.0 code, attribute
lists are no longer than 4.

Fetching via M:module_info/1, however, the list is flat, and can be quite
long, mostly depending on the number of types. Looking again at OTP, but
loading each module and fetching the attributes via M:module_info/1 (the
example below lists all modules with at least 20 attributes):

19> lists:foldr(fun(D,Acc) -> filelib:fold_files(D, ".*", false, fun(F,A)
-> M = list_to_atom(filename:basename(F,".beam")), try
M:module_info(attributes) of As -> case length(As) of L when L >= 20 ->
[{M,L}|A]; _ -> A end catch error:_ -> A end end, []) ++ Acc end, [],
code:get_path()).
[{gen_fsm,20},
 {erl_parse,208},
 {xmerl_xpath_parse,45},
 {xref_parser,34},
 {snmpc_mib_gram,232},
 {megaco_text_parser_v3,413},
 {megaco_text_parser_v2,378},
 {megaco_text_parser_v1,325},
 {megaco_text_parser_prev3c,410},
 {megaco_text_parser_prev3b,396},
 {megaco_text_parser_prev3a,389},
 {megaco_text_mini_parser,25},
 {icparse,185},
 {edoc_parser,94},
 {diameter_dict_parser,76},
 {cosNotification_Grammar,36},
 {core_parse,224}]


BR,
Ulf W

2017-09-28 8:56 GMT+02:00 Max Lapshin <max.lapshin@REDACTED>:

> Hi.
>
> We have moved from plain text lager logging to our own event system that
> has main difference: we have limited amount of possible events that are
> compiled in code.
>
> In known_events.erl I have:
>
>
> -event({stream_started, info, ["stream started"]}).
>
> and then I write in code:
>
>
> events:stream_started([{media,Name}]).
>
> Later all this is printed (using lager formatting, it is cool).
>
>
>
> So we are close to the problem.  My code that was handling such:
>  events:stream_started was searching for predefined event via:
>  known_events:module_info(attributes).
>
> Suddenly eprof told that I need to look carefully at this place and I
> wrote parse_transform that takes all attributes from module AST and
> compiles them into code. Also I've generated function
>  known_events:find(Name) with all possible clauses.
>
> On single core of my macbook I got 4000 times speedup.  100 000 of fetches
> known_events:all()  took about 8 seconds with module_info(attributes) and
> about 2 milliseconds with generated and compiled code.
>
>
> So lesson from here is: try to avoid calling module_info(attributes) in
> innner loops. It may be suprisingly expensive.
>
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20171001/ace9ee7c/attachment.htm>