Erlang on openstack VM with numa awareness

[Ameretat Reith]

On Fri, Feb 7, 2020, 4:47 PM Satish Patel <satish.txt@REDACTED> wrote:

> My openstack compute host with 32 core so i have created 30 vCPU core
> vm on it which has all NUMA awareness, when i am running Erlang
> application benchmark on this VM getting worst performance but then i
> create new VM with 16 vCPU core (In this case my all VM cpu pinned
> with Numa-0 node) and in this case benchmark result was great.
>

I don't know about how OpenStack utilize NUMA, but I had same experience
benchmarking QEMU VMs; best VMs were the ones all vCPUs pinned to cores on
single host NUMA node. Next to them, were VMs that half CPUs pinned to
cpuset in one NUMA node and other to another node, Of course I set proper
NUMA topology by QEMU args. Worst performance were VMs with no
pinning/affinity setting which means, QEMU defined two NUMA nodes (as host)
while vCPUs getting swapped over different cores on different real NUMA
nodes. I see it expectable. My tests were targetting nothing Erlang based,
it was redis and some other things.

>
> But interesting thing is when i run same erlang application run on
> bare metal then performance is really good, so trying to understand
> why same application running on VM doesn't perform well?
>

I think because Erlang is smart about NUMA but in virtualized environment,
its knowledge about NUMA nodes are not reliable without CPU pinning.

>
> Is there any setting in erlang to better fit with NUMA when running on
> virtual machine?
>

I don't believe it's something Erlang could offer. I would just test
pinning VM vCPUs to different CPU sets based their NUMA node and set proper
topology on VM. Then i expect Erlang utilize NUMA nodes as smart as it does
on bare metal.

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20200207/15b224bf/attachment.htm>