What does scheduler/aux mean in msacc?

Jesper Louis Andersen jesper.louis.andersen@REDACTED
Tue Jul 7 12:40:02 CEST 2020


You should run on the same machines, and preferably at the same time (back
to back). When you are running under a hypervisor, you are cooperating with
other users of the hardware, and they can wreak serious havoc on your
benchmarks. I've seen 20% fluctuations in performance on AWS hardware due
to this. The usual problems stems from how the CPUs handle caches and if
there are other users who move a lot of data around, you will see a lower
performance due to this.

My advice will be to reconstruct the problem on local hardware. If you
cannot do this, then chances are you are looking at some AWS artifact
rather than a change in OTP behavior.



On Mon, Jul 6, 2020 at 5:14 PM Fernando Benavides <elbrujohalcon@REDACTED>
wrote:

> Yeah… the curious thing is that both machines are the same kind of AWS
> virtual machines… And I had 2 running OTP22 and 2 running OTP21… and both
> OTP22 machines showed xen_hypercall_event_channel_op while both OTP21
> didn't (actually… they did, but with a very smaller % of overhead). And,
> BTW, the same happened with smp_call_function_many (more overhead
> associated with that function in OTP22 than OTP21).
>
> I'll keep digging, but if anybody has any idea of what to look for, I
> would appreciate it a lot.
>
> Thanks
>
> On Mon, Jul 6, 2020 at 5:05 PM Max Lapshin <max.lapshin@REDACTED> wrote:
>
>> First post from the internet about xen_hypercall_event_channel_op:
>> https://www.scylladb.com/2018/06/21/impact-virtualization-database/
>>
>> TL;DR: metal server better than virtual
>>
>> On Mon, Jul 6, 2020 at 5:26 PM Fernando Benavides <
>> elbrujohalcon@REDACTED> wrote:
>>
>>> I'll add another data point…
>>> As Lukas suggested, I used *perf* on the running systems and this is
>>> what I've got…
>>>
>>> For OTP22:
>>>      2.33%  8_scheduler      beam.smp             [.] process_main
>>>
>>> *     …the rest of the schedulers running process_main…*     1.57%
>>>  6_scheduler      beam.smp             [.] process_main
>>>      0.34%  6_scheduler      [kernel.kallsyms]    [k]
>>> xen_hypercall_event_channel_op
>>>      0.32%  7_scheduler      [kernel.kallsyms]    [k]
>>> xen_hypercall_event_channel_op
>>>      0.25%  14_scheduler     beam.smp             [.] copy_struct_x
>>>      0.25%  2_scheduler      beam.smp             [.] copy_struct_x
>>>      0.25%  9_scheduler      [kernel.kallsyms]    [k]
>>> xen_hypercall_event_channel_op
>>>      0.23%  11_scheduler     [kernel.kallsyms]    [k]
>>> xen_hypercall_event_channel_op
>>>      0.22%  11_scheduler     beam.smp             [.] copy_shallow_x
>>>      0.22%  9_scheduler      libc-2.17.so         [.]
>>> __memcpy_ssse3_back
>>>      0.20%  13_scheduler     [kernel.kallsyms]    [k] __lock_text_start
>>>      0.20%  3_scheduler      [kernel.kallsyms]    [k]
>>> xen_hypercall_event_channel_op
>>>      0.19%  10_scheduler     beam.smp             [.] erts_cmp_compound
>>>
>>> For OTP21:
>>>      2.45%  4_scheduler      beam.smp             [.] process_main
>>>
>>> *     …the rest of the schedulers running process_main…*     1.91%
>>>  5_scheduler      beam.smp             [.] process_main
>>>      0.45%  15_scheduler     [kernel.kallsyms]    [k] __lock_text_start
>>>      0.27%  10_scheduler     [kernel.kallsyms]    [k] __lock_text_start
>>>      0.27%  3_scheduler      [kernel.kallsyms]    [k] __lock_text_start
>>>      0.27%  7_scheduler      [kernel.kallsyms]    [k] __lock_text_start
>>>      0.25%  1_scheduler      beam.smp             [.] copy_struct_x
>>>      0.25%  2_scheduler      [kernel.kallsyms]    [k] __lock_text_start
>>>      0.23%  2_scheduler      beam.smp             [.] copy_struct_x
>>>      0.23%  7_scheduler      beam.smp             [.] copy_struct_x
>>>      0.20%  4_scheduler      beam.smp             [.] copy_struct_x
>>>      0.20%  6_scheduler      [kernel.kallsyms]    [k] __lock_text_start
>>>      0.18%  14_scheduler     beam.smp             [.] copy_struct_x
>>>
>>> I'm not sure what any of those functions are, or why
>>> xen_hypercall_event_channel_op is so much popular in OTP22, tho… ��
>>>
>>> On Mon, Jul 6, 2020 at 3:42 PM Max Lapshin <max.lapshin@REDACTED>
>>> wrote:
>>>
>>>> In a nutshell there is a non-reproducible fluctuation of measurement:
>>>> one subsystem takes 10% instead 8%, another 21% instead 23%.
>>>>
>>>> Totally there seems to be no change.
>>>>
>>>> It is extremely hard to make reliable measurement that helps to find 8%
>>>> to 10% change.
>>>>
>>>
>>>
>>> --
>>>
>>> <https://about.me/elbrujohalcon?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=gmail_api&utm_content=thumb>
>>> Brujo Benavides
>>> about.me/elbrujohalcon
>>> <https://about.me/elbrujohalcon?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=gmail_api&utm_content=thumb>
>>>
>>
>
> --
>
> <https://about.me/elbrujohalcon?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=gmail_api&utm_content=thumb>
> Brujo Benavides
> about.me/elbrujohalcon
> <https://about.me/elbrujohalcon?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=gmail_api&utm_content=thumb>
>


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20200707/d3096078/attachment.htm>


More information about the erlang-questions mailing list