[erlang-questions] Millions of processes?

YourSurrogateGod yoursurrogategod@REDACTED
Mon Sep 29 02:37:57 CEST 2008


Purely out of curiosity, I'd like to read the said articles (if you
have the links for them). I have a rought idea how micro-kernels work
(mostly from Andrew Tannenbaum's website) and I'm fascinated by them.

On Sep 24, 6:23 am, Bengt Kleberg <bengt.kleb...@REDACTED> wrote:
> Greetings,
>
> It is not only _ideas_ in computer science that are held in contempt (as
> quoted below). Ages ago (before 2000) I read an article about Linux in
> embedded environments. The article quoted Linus Torvalds on why not to
> use micro kernels. The reasons where that they are:
> 1 Experimental
> 2 Complex
> 3 Slow
>
> After looking around for a while I found plenty of articles about
> commercial micro kernels, and benchmarks showing micro kernels running
> workloads faster than monolithic kernels. So 1 and 3 seemed to be
> incorrect.
> I submitted these findings to the magazine, which prompted an answer
> from Mr Torvalds. He assert that all three where true, but did not
> discuss what I had found. So (IMHO) it is also facts that are not held
> in very high regard.
>
> bengt
>
>
>
>
>
> On Tue, 2008-09-23 at 19:09 +0200, Ulf Wiger (TN/EAB) wrote:
> > NPTL is fast, but AFAIK uses a minimum stack size of 8KB
> > per thread (the minimum heap size for erlang processes
> > seems to be 932 bytes on a 32-bit system). There also seem
> > to be other limits, making it very difficult in practice
> > to reach anywhere near 100,000 threads, and it's not
> > encouraged either.
>
> >http://nptl.bullopensource.org/Tests/NPTL-limits.html
>
> > In the Linux kernel FAQ, the philosophy on threads is
> > explained thus:
>
> > "Avoid the temptation to create large numbers of threads in your
> > application. Threads should only be used to take advantage of multiple
> > processors or for specialised applications (i.e. low-latency real-time),
> > not as a way of avoiding programmer effort (writing a state machine or
> > an event callback system is quite easy). A good rule of thumb is to have
> > up to 1.5 threads per processor and/or one thread per RT input stream.
> > On a single processor system, a normal application would have at most
> > two threads, over 10 threads is seriously flawed and hundreds or
> > thousands of threads is progressively more insane.
> > A common request is to modify the Linux scheduler to better handle large
> > numbers of running processes/threads. This is always rejected by the
> > kernel developer community because it is, frankly, stupid to have large
> > numbers of threads. Many noted and respected people will extol the
> > virtues of large numbers of threads. They are wrong. Some languages and
> > toolkits create a thread for each object, because it fits into a
> > particular ideology. A thread per object may be appealing in the
> > abstract, but is in fact inefficient in the real world. Linux is not a
> > good computer science project. It is, however, good engineering.
> > Understand the distinction, and you will understand why many widely
> > acclaimed ideas in computer science are held with contempt in the Linux
> > kernel developer community. "
>
> >http://www.kernel.org/pub/linux/docs/lkml/#s7-21
>
> > BR,
> > Ulf W
>
> > Zvi skrev:
> > > I'm no Linux expert, but
>
> > >http://en.wikipedia.org/wiki/Native_POSIX_Thread_Library
>
> > > "The Native POSIX Thread Library (NPTL) is a software feature that
> > > enables the Linux kernel to run programs written to use POSIX Threads
> > > fairly efficiently. In tests, NPTL succeeded in starting 100,000
> > > threads on a IA-32 in two seconds. In comparison, this test under a
> > > kernel without NPTL would have taken around 15 minutes."
>
> > > I guess future Erlang VM will offer some more generic MxN threading
> > > model, i.e. M Erlang user-level processes implemented on N
> > > "schedulers" - native threads. Today in SMP Erlang is only limited
> > > support (i.e. command line options) to specify number o scheduler and
> > > no programmatic support for affinity of schedulers per core and
> > > Erlang processes per schedulers.
>
> > > Zvi
>
> > > Bob Ippolito wrote:
> > >> We've got a couple applications that use thousands of processes per
> > >>  node. If those were pthreads, we'd be out of RAM before actually
> > >> doing anything.
>
> > >> 2008/9/23 Bard Bloom <ba...@REDACTED>:
> > >>> I've seen in Erlang promotional materials some rather impressive
> > >>> claims about how cheap Erlang processes are, and how many of them
> > >>> one can spawn. Which is pretty cool. But, what Erlang programs
> > >>> take advantage of that kind of power? Are there any examples of
> > >>> programs which use huge numbers of processes in interesting ways?
> > >>> (I am the local Erlang fancier. I got challenged on that point,
> > >>> and didn't have a very good answer.)
>
> > >>> Thanks very much, Bard Bloom
>
> > >>> _______________________________________________ erlang-questions
> > >>> mailing list erlang-questi...@REDACTED
> > >>>http://www.erlang.org/mailman/listinfo/erlang-questions
>
> > >> _______________________________________________ erlang-questions
> > >> mailing list erlang-questi...@REDACTED
> > >>http://www.erlang.org/mailman/listinfo/erlang-questions
>
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questi...@REDACTED
> >http://www.erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questi...@REDACTED://www.erlang.org/mailman/listinfo/erlang-questions



More information about the erlang-questions mailing list