[erlang-questions] Question about message passing paradigm

[Edwin Fine]

On Tue, Jul 1, 2008 at 8:20 PM, Richard A. O'Keefe <ok@REDACTED>
wrote:

>
> On 1 Jul 2008, at 7:37 pm, Edwin Fine wrote:
>
>> Thank you for a very interesting and informative analysis. I must admit
>> that I tend to lump together Erlang/OTP as "Erlang" and see solutions in
>> that context. In a way, I feel as if using Erlang "standalone" without OTP
>> is very roughly analogous to using C++ without the STL: it can be done, but
>> why on earth would one want to?
>>
>
> Because the STL
>  - is still not fully portable between compilers;
>   in theory it should not be, but it takes you into
>   deep template territory where compilers have
>   incompatible bugs (though they are improving)
>  - is in my experience less efficient than home-brew
>   code (the STL relies, like Haskell, on *serious*
>   optimisation which compilers do not always do)
>  - gets you some of the most incomprehensible
>   compiler error messages (un)imaginable when you
>   make mistakes, which you always do, because
>  - it is unpleasantly complex to use (this will, I
>   hope, be remedied in C0x, at least if I can
>   trust Stroustrup's summary of what's going to be
>   there).
>

Maybe after using it for a long time, it's doesn't seem that complicated. At
least, not compared to C++ itself. C++ itself is IMHO horribly complex and
arguably a language to be used only by C++ experts. Just writing
exception-safe code alone can be dreadfully difficult, never mind the
powerful yet labyrinthine maze that is called templates. I've now used the
STL extensively for many years, and it has a large advantage in that it is
more or less supported by all compliant compilers (and "free" as in beer).
It pretty much does the trick. Initially, the ability of various compilers
to handle templates was pretty bad and inconsistent so I rolled my own
portable libraries. I had to write a significant amount of commercial C++
code that had to be ported across OS/390, HP/UX, Windows, Solaris, AIX, and
Linux, so I have a feel for the issues involved. I found it amusing that
Andrei Alexandrescu could write syntactically valid template code that no
C++ compilers could parse at that time. After a while, though, the effort of
maintaining (testing, documenting, fixing) all that code across all those
systems became too much and I used the STL exclusively after that. I had to
choose a lowest common denominator approach and avoid doing anything really
fancy, but it was still worth it.
The STL was efficient enough for most purposes. In the IT department of a
large telco where I worked for some years, trying to improve quality and
establish standards, the code was so inefficient that the benefits of using
the STL far outweighed any efficiency concerns. They didn't even compile the
code with optimization when it went into production, so the overhead of
those templates must have been quite significant, yet it didn't even
register on the radar compared to the big concerns (like, actually getting
the code to work, and getting the developers to understand why it's slow to
do a linear lookup in a vector of 100,000 items, in a loop that iterates
50,000 times). When you're in the trenches, you are grateful for anything
that works as advertised most of the time. Gladly, I am out of the C++
business right now and happily learning the Erlang world, which is a joy in
comparison.


> None of these applies to OTP, but the argument that
>
>  - it is another large and complex body of material
>   to master on top of something itself unfamiliar.
>
> does apply to both.


Agreed.


> I have a colleague who has done
> serious commercial software development in C++ and
> flatly refuses to have anything to do with the STL
> (see reasons above).


I did, too, until a few years ago (see above).


>
>
> Let's take the current example and see if we can squeeze a
> bit more out of it.  Tim Watson pointed out that there is
> an issue about "locking" and failing processes.  My take
> on this is "let it crash": when processes are coupled like
> this they should be as a rule be linked and if one dies,
> all should die.  The OTP behaviours take care of that kind
> of thing, which is why *once you have grasped the basics*
> you should try them before rolling your own.  What's really
> interesting here is that the original system was written in
> terms of threads and locking.  Have you looked at thread
> interfaces lately?


Unfortunately, I have worked a significant amount with pthreads, Java
threads, Windows threads, and OS/2 threads, and consider that skill set to
be in the same domain as C++: for experts (as opposed to the "average"
developer found in IT shops and who knows how many software companies) only.
The fact that we are now seeing multicore processors and are being exhorted
to write more thread-bearing code makes me shudder. I suspect that most of
the readers of this newsgroup are highly skilled developers, hopefully
working for great companies, and I wonder if they have been exposed to the
horrors of the "average" developer (as evidenced in "The Daily WTF" and
similar web sites). I hope I am not coming across conceited; I'm not like
that at all. It is just that I have spent 25 years working hard to be as
good a software guy (for want of a better description) as possible, and for
my troubles have almost always been put in a position where I have had to
teach and mentor the "average" developer (and fix literally millions of
lines of code, usualy C++). My experience (and others I am sure will have
differences of opinion) is that the level of the average developer is
trending steeply downwards as developers become a globalized commodity
("resource") and compete more on price than on ability. These days, I find
it rare to see halfway decent code written in ANY language, and the thought
of zillions of lines of thread-bearing C++ and Java software being let loose
on the world by people who can't even spell O(N^2) scares the heck out of
me. Sorry if I am ranting a bit.


> [Single Unix Specification version 3]
>
> pthread_mutex_lock(&mutex)
> pthread_mutex_unlock(&mutex)
>
> There are four kinds of mutex in POSIX threads:
>  PTHREAD_MUTEX_NORMAL
>        recursive locking deadlocks
>        unlocking a mutex that is already unlocked
>        or that is locked by another => undefined
>  PTHREAD_MUTEX_ERRORCHECK
>        these conditions return an error code
>  PTHREAD_MUTEX_RECURSIVE
>        recursive locking works
>        unlocking a mutex that is already unlocked
>        or that is locked by another => error code
>  PTHREAD_MUTEX_DEFAULT
>        same as PTHREAD_MUTEX_NORMAL
>  Anything else
>        All behaviour undefined
>
> [Solaris 2.10]
> If
> (1) _POSIX_THREAD_PRIO_INHERIT is defined
> (2) the mutex was initialised with protocol
>    attribute PTHREAD_PRIO_INHERIT
> (3) the mutex was initialised with robustness
>    attribute PTHREAD_MUTEX_ROBUST_NP
> (4) the last holder of the lock crashed
> then
> (A) an attempt to lock the mutex will 'fail'
>    with the error code EOWNERDEAD
> (B) but in fact the attempt will have succeeded
> (C) it is up to the *new* owner of the lock to
>    try to clean up the state
> (D) if it can, it calls pthread_mutex_consistent_np
> (E) if it crashes, the next locker will get the
>    same error code and the same chance to recover
> (F) if it can't, it should unlock the mutex, and
>    future lockers will get a *different* error code
>    (ENOTRECOVERABLE).
> (G) it is possible to call pthread_mutex_consistent_np
>    on mutexes that aren't held or didn't need
>    recovery and the behaviour is undefined
>
> If a mutex with the default PTHREAD_MUTEX_STALLED_NP
> robustness value is held by a thread that dies,
> future locks are "blocked in an unspecified manner".
> What this means in practice I'm not sure.
>
> If you reckon the Solaris 2.10 treatment of crashed
> lock holders is a mess, perhaps you can point me to
> something better in SUSv3.  I can't find anything in
> SUSv3 to say _what_ happens when a lock owner crashes.
> (And don't get me onto the subject of cancellation points.)
>
>
> What kind of building block is _this_ for building
> reliable systems?
>

One made of clay and straw and baked in the sun.


>
> Message passing plus linking is so much easier to
> have justified confidence in that pthreads and TBB start
> to look like extremely sick jokes.
>

I couldn't agree with you more. One thing that amazed me about Erlang is
how, once I had figured out how to write something, it Just Worked (TM). I
fell in love.


>
>
>  <off-topic>One last thing: I read the Ethics of Belief after poring over
>> one of your posts recently, and was exceptionally impressed with the
>> gentleman's writing and philosophy, with which I strongly agree. In that
>> regard, I'd like to misquote John Stuart Mill, namely, "A foolish certainty
>> is the hobgoblin of little minds". (Actually, I think my version is a slight
>> improvement ;) I am unfortunately seeing the mechanism of insufficiently
>> examined beliefs at work today, resulting in the persecution of a friend of
>> mine by way of (the almost totally belief-based) Shaken Baby Syndrome. So
>> the essay really resonated deeply with me. I daresay William K. Clifford
>> would have had a lot to say about this. I wish he were still alive to do so.
>> Bertrand Russel too.</offtopic>
>>
>
> <also-off-topic>Clifford takes a really good idea and pushes it beyond the
> bounds
> of reason; he manages, presumably without intending to, to make any belief
> in
> science ethically unjustifiable.  Specifically and in Clifford's day
> topically,
> Clifford's rule about believing other people meant that it would have been
> *bad*
> for anyone to believe in Natural Selection.  I encountered Clifford's paper
> in
> a book by DeMarco and someone else about risk management in software
> engineering.
> I find the idea that Shaken Baby is still credited deeply upsetting; please
> convey
> my sympathy and good wishes to your friend.
> </also-off-topic>
>

<rant>
I won't go off-topic after this, but I just wanted to write two things:

(a) Shaken Baby is sadly alive and persecuting parents and child carers in
the USA, Australia and England.
(b) After reading dozens of papers on the SBS myth, I mean syndrome, and
seeing the terrible cost to innocent families due to junk science, I rather
lean toward being too much a "Cliffordite" than too little. This is
particularly true after I read of scientific bias and just plain fraud (all
that lovely research money is so enticing). I actually took Clifford's
advice by following up on as many references as I could find in
peer-reviewed papers, and I was appalled at how, in many cases, results were
used selectively from these papers to back up shaky claims and make them
look solidly research-based. These days I am almost reluctant to accept the
evidence of my own eyes, having learned how deeply subjective "reality" is,
and how what is up today is down tomorrow. Galileo and many others were
persecuted for being "heretical" and later shown to be correct. Great "laws"
of nature (classical relativity -> special relativity -> general relativity
-> quantum mechanics -> string theory -> ??) were challenged and either
overturned or radically transformed by those brave and noble enough to
endure scorn and criticism. Heretics were burned at the stake not that long
ago; are we humans really any different now? Yes, I like Clifford. Rather be
too critical than not critical enough, even if it risks making beliefs in
science ethically unjustifiable: in the days of the Internet and easy claims
of anything, and people who will believe almost any plausible claims, we
need to be more vigilant than ever.</rant>


-- 
The great enemy of the truth is very often not the lie -- deliberate,
contrived and dishonest, but the myth, persistent, persuasive, and
unrealistic. Belief in myths allows the comfort of opinion without the
discomfort of thought.
John F. Kennedy 35th president of US 1961-1963 (1917 - 1963)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20080701/15f32a0a/attachment.htm>