[erlang-questions] Some facts about Erlang and SMP

[Matthew Sackman]

On Tue, Sep 16, 2008 at 11:34:03AM +0200, Kenneth Lundin wrote:
> - The SMP VM with only one scheduler is slightly slower than the non
> SMP VM. The SMP VM need to to use all the locks inside but as long as
> there are no lock-conflicts the overhead caused by locking is not
> significant (it is the lock conflicts that takes time). This explains
> why it in some cases can be more efficient to run several SMP VM's
> with one scheduler each
> instead on one SMP VM with several schedulers. Of course the running
> of several VM's require that the application can run
> in many parallel tasks which has no or very little communication with
> each other.

I'm having some difficulties with the "it's only when locks conflict
that locking time is significant" idea. My understanding with the
pthreads locks is that you start with a CAS, and if that fails then you
spin for a while (with CAS), and if you spin for too long then you go a
kernel lock. If that's wrong then I'm sorry, as most of the following
will also be wrong!

Now assuming the above is correct (or at least close enough for jazz),
in the conflict case, sure, you're going to block, which is the point,
and then some time in the future either the kernel will wake you up
(pretty slow), or the spinning finally succeeds in which case the
wakeup should be really fast.

If there's not a conflict then the first CAS will succeed. But a CAS is
many hundreds of cycles as you've got to do cache-coherency and then
really go all the way out to RAM and load the values in fresh,
especially in a multisocket box. So hundreds of cycles does not seem
like "not significant", at least to me. Or have I missed something
obvious here?

I rather expect that you'll end up with lock-free per-cpu task-queues
with workstealing algorithms which as a colleague has just pointed out,
has been used for these purposes for some time (at least 10 years);
though I do appreciate the "get it out working and then tune it"
approach.

Matthew