time for a truly threaded Erlang?

[Jani Launonen]

>But is there any need for a truly threaded Erlang? The native Linux
>threads (in the current version) are pretty heavy-weight, and not much
>different from full-fledged processes (indeed, the "fork" syscall in Linux
>is just a specialisation of "clone" which is used to create kernel
>threads). The cited posting mentions some improvements in "clone", but
>the relatively high cost of thread creation (compared to Erlang processes)
>would probably still be an issue.
>
>Multiple kernel threads can be useful only on multi-processor machines, so
>they can be scheduled to different CPUs. But even in that case you don't
>need 100,000 threads -- the reasonable number of them is bounded by the
>number of CPUs available. Erlang processes are so light-weight and
>efficient that it makes much more sense to use them, rather than kernel
>threads, as units of concurrency...

I think, Ulf was promoting the kind of Erlang VM that Peter Hedqvist
studied (http://www.erlang.se/publications/xjobb/0089-hedqvist.pdf) and
made his thesis about. An Erlang VM that can take full advantage of SMP
machines without deploying an EVM per CPU would be very beneficial. There
would be no need to make own load balancing as OS would schedule threads
anyways.

So when one needs some serious throughput (as Eric Newhuis seems to need),
it would be very useful to scale to several CPUs with only one EVM running,
without the need to balance load across several nodes by static or dynamic
load balancing. One could, of course, start several EVMs and have one EVM
run the NotificationChannel process, other some subset of ConsumerAdmins
and so on, but the load balancing would propably not be as good as what OS
could do.

So wouldn't it be more effcient to have one EVM running several threads
(running Erlang processes) on one SMP machine, where processes / threads
does get scheduled by OS and message sending could be optimised by shared
heap than doing distribution through TCP/IP stack or pipe and load
balancing on top of that. Every message from one EVM to another have to
across the (heavy) process boundary, which isn't nice for little and
frequent messages (I haven't really measured, but it's just my gut
feeling).

>Leonid Timochouk
>Computing Laboratory
>University of Kent at Canterbury

With respect,

-+-+-+-
Jani Launonen
Student. . . . . . . . . .University of Oulu, Dept. of EE
Assistant Researcher . . .Apricot Project

"Computing is a field which has one of the shortest collective memories
of any engineering or scientific discipline." - Marty Fouts, comp.distributed