[erlang-questions] : Subtle behaviour of Erlang scheduler

[Kenneth Lundin]

See comments inline.

On 5/29/07, KatolaZ <me@REDACTED> wrote:
> On Mon, May 28, 2007 at 10:52:58PM +0200, Ulf Wiger (TN/EAB) wrote:
> >
> > Corrado Santoro wrote:
> > >
> > > But this does not mean that, in my opinion, the Erlang
> > > priority system has a serious flaw and should be rethought.
> > > Probably, in a telecommunication system, priorities could not
> > > be so important, but if you use Erlang in real-time embedded
> > > environment (and it's argued that Erlang is for
> > > soft-real-time systems),
> >
> > Indeed, telecommunication systems are soft-real-time,
> > which means that it is sufficient to meet deadlines
> > in a high percentage of the cases, but missing a
> > deadline is not considered a fatal error.
> >
> > Erlang shines in this regard, IMO, but in my experience,
> > process priorities don't do much to help in this type
> > of system. Thus the recommendation to use 'normal'
> > priority in the vast majority of cases.
>
> Well, guys, I thought Erlang was really going to be used in many
> different application fields, from telecommunication platforms to web
> apps, from massively concurrent systems to distributed high
> performance applications.

YES, Erlang is already successfully used for this and the use for this
kind of applications will most probably increase taking the SMP
support and the
easy way to utilize multi core CPU's into account.

>
> I also dreamed about Erlang as a real, affordable, safe and robust
> alternative to many ugly and still widely-used languages and
> platforms, such as C#/.net and Java, or at least I understood so,
> attending the last two Erlang User Conferences in 2005 and 2006.
> I also started to talk about erlang at university courses.....

Also a correct observation from your side.
Very good that you started to talk about Erlang at university courses it is
inline with the common opinion among SW research that functional languages
Like Erlang, Haskell etc. have advantages because of their higher
abstraction level
compared with languages like C, C++ when it comes to multithreaded
programming, which is needed in order to utilize multi  core CPU's.

>
> But the last two mails, which suddenly close the Pandora Vase we
> opened looking a little deep inside the process scheduler, simply
> explain that Erlang developers (or at least Ericsson OTP group), has
> little or no interest on having a real robust, affordable,
> distributed, secure and real-time platform.

Erlang was originally designed to be an efficient development environment for
rubust, parallel possibly distributed control systems with soft
real-time requirements. Erlang fulfill that purpose very well.
The erlang distribution protocol is designed to be used in a local
network and lacks many security mechanisms that might be needed if
used over the public internet.
(If that is what you mean with secure?).

Supporting 'hard' real-time is a completely different thing which would require
a major redesign of the virtual machine possibly also many new
primitives and changes to language.

To start with Erlang is most typically run on an operating system
(Linux, Solaris,Windows) which is
not categorized as a real time operatring system.
So real-time characteristics of Erlang can not be better than what the OS offers
to the OS process and threads that the Erlang VM executes in.

In my view however the bulk of SW used in applications don't need
"hard" real-time
characteristics and the small fractions that need can be implemented
as separate processes, drivers etc. which are then interfaced from
Erlang (provided that
the OS supports "hard" realtime to som e extent.

>
> After we discovered a big bug in the scheduler, and reported it to the
> community, the only answer we got was: we don't use that scheduler
> code in OTP, this bug does not heavily affect OTP, processes in
> telecom apps usually use normal priority and since telecom apps are
> our main business and represent 90% of actual Erlang code, it does not
> matter for us.
>
As already explained you have not found a big bug.
You have found something that needs to be explained better in the docs and
that the behaviour of the system with regards to priorities could
perhaps be improved if it becomes evident that it is something that
give problems to many users.

I try to remember how many issues or questions regarding process
priorities we have had over the last 10 years and it is less than a
handful (this thread included).
So it seems not very disturbing for the majority of users.

> Well, maybe it does not matter for telecom apps, maybe it does not
> matter for 90% of Erlang code, maybe it is a so rarely used feature,
> but ERLANG HAS A BUGGY SCHEDULER, EVEN IF YOU DON'T WANT TO ADMIT IT.

See Rickard Greens explanation about the scheduler.


/Regards Kenneth (Erlang/OTP responsible within Ericsson)