OTP22 performance on lists

[Dániel Szoboszlay]

Hi Brujo,

I tried to reproduce your results, and for me OTP 22 seems to be
consistently faster for rec and about the same for lrec:

Test case     Min         Max        Median     Average
rec  / OTP 21 124.170us   305.990us  127.120us  128.688us
rec  / OTP 22  62.570us   289.190us   67.610us   68.976us
lrec / OTP 21  16.110us    22.570us   17.790us   17.881us
lrec / OTP 22  15.390us    22.710us   17.170us   17.441us

(These are times for single invocation of the functions for much smaller
inputs than in your example.)

However, there are some OTP 21 versions where due to a configuration bug
the resolution of the native clock was much lower. For example
erlang:convert_time_unit(1000,
native, microsecond). normally gives 1 on my machine (so the native clock
has nanosecond resolution), and on the buggy OTP 21 versions I get 610. On
these OTP 21 versions I see that your rec benchmark seems to run ~6% faster
than with OTP 22, but this is measurement error due to the lower clock
resolution.

Could you please check you have the same clock resolution on both OTP
versions?

Cheers,
Daniel

On Wed, 1 Jul 2020 at 12:56, Fernando Benavides <elbrujohalcon@REDACTED>
wrote:

> (Sorry if you receive this email twice, I sent it from another account and
> it seems like it didn't reach the mailing list then)
>
> Hi erlangers,
>
> Yeah! It's me talking about performance, you read that subject right ��‍♂️
> Anyway… At NextRoll we're in the process of migrating our systems from
> OTP21 to OTP22 (not 23, yet) and our tests showed a *huge* impact on
> performance in general, that was not associated to anything in particular.
>
> While trying to figure out what was causing it, we came up with a very
> very basic example of things that are consistently slower in OTP22.
>
> I created this gist to show it:
> https://gist.github.com/elbrujohalcon/d4e995fbc4b93fadddfd1f0d6b9f8121
>
> I'm aware that these kinds of micro-benchmarks are treacherous and they
> may vary wildly depending on context. Nevertheless, on both on one of our
> servers in AWS and on my machine (A MacOS Pro running Catalina 10.15.5) and
> using kerl to install multiple versions of OTP 21 and 22… every single time
> I run the tests I found the same results…
>
> Always starting the nodes with *erl -boot start_clean *then running the
> following in both OTP21 and OTP22…
>
> c(test), test:bench({test, lrec}, 250, 5000, 2000).
>
> …generates very similar numbers in both versions, regardless of the
> numbers used for the different parameters. But…
>
> c(test), test:bench({test, rec}, 250, 5000, 2000).
>
> …consistently generates larger results in OTP22 than OTP21. I tried with
> different values for the number of tests, the number of lists and their
> length and sometimes the difference is more evident, sometimes less… but
> OTP22 times are *always* larger.
>
> I found this in the OTP22 readme…
>
>   OTP-15427    Application(s): erts
>                Appending lists (The ++ operator) will now yield
>                properly on large inputs.
>
> So… questions…
> 1. Has anybody experienced (and hopefully solved) this problem before when
> migrating to OTP22?
> 2. Do you think OTP-15427 can be related to what I'm seeing?
> 3. Can someone confirm if you also experience the same difference in
> performance when running the same benchmarks that I pasted on that gist?
> 4. Is there anything else I should try/test/use to check?
> 5. Am I going slightly mad?
>
> Thanks in advance, cheers :)
>
>
> --
>
> <https://about.me/elbrujohalcon?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=gmail_api&utm_content=thumb>
> Brujo Benavides
> about.me/elbrujohalcon
> <https://about.me/elbrujohalcon?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=gmail_api&utm_content=thumb>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20200702/351e46f3/attachment.htm>