OS scheduling vs. VM scheduling

[Joachim Durchholz]

todd wrote:

> There is also a cultural issue thay may come up with threads
> is that many if not most programmers are conditioned against threads.

That's largely because the cost for starting threads is so high.
(Second on the list are the debugging difficulties.)

The first is going to go away.
The second is less of an issue with Erlang, and I think people will 
recognize that.

> I have run into this over and over. If your linux box suddenly had
> 100,000 threads a lot of people would freak. Hiding them
> in erlang processes is a quieter approach :-)
> 
> Plus, i would test the hell out if it before i believed them...

Agreed.

> Ulf Wiger wrote:
> 
>> Note that they're talking about threads, i.e. just creating
>> a context in shared memory.

I seem to recall it was about processes, but I may be wrong. Or maybe 
the journalist got it wrong :-)

 >> Erlang "threads" do not share
>> memory (conceptually, even if they do in reality). This is
>> something to think about if one wants to share the memory
>> space with other languages -- Erlang processes _behave_ as if
>> their memory is protected and separate from others.

That's usually handled via the usual warnings in the interoperability 
documents. I think that C code will respect this; and if there's any 
danger of having a problem here, the Erlang side could always hand out 
copies of the data instead of giving out the data itself.
Actually this is the least of the worries invoved, talking to C or any 
other language will usually involve a conversion anyway. If the Erlang 
side doesn't rely on the foreign-format data being unchanged (and why 
should it?), there's nothing that can go wrong.

>> Another implication is that the VM has to be made thread-safe.
>> It is not so today. The upside is that it would become possible
>> to take advantage of multipro architectures in a wholly different
>> way than today.

Exactly.

>> The details that appear to have changed since then are the
>> number of threads that can be created and the cost of creating
>> them (but basically only on Linux.)

Currently, yes.

However, that scheduler is a very sexy feature. The advice to keep the 
process count down permeates the Unix sysadmin literature; with the O 
(1) scheduler, this is no longer applicable, so entirely new ways of 
setting up Linux servers become sensible.
So I expect that the various Unix vendors will pick that lead up pretty 
quickly, either voluntarily or under market pressure. Given that the 
code already exists, they'll probably be able to steal shamelessly (the 
algorithms if not the code), so it's going to take relatively little 
effort on their side; and with that "approved by Linus" seal on that 
software, they'll take the approach more seriously than research 
projects from academia.

Of course, my magic predictive mirror is as distorted and cobwebby as 
that of anybody else :-)

Regards,
Jo
--
Currently looking for a new job.