(Prolog + LISP + Erlang) with integration issues versus C++

[Richard A. O'Keefe]

Dev Functional <devfunctional@REDACTED> wrote:
	Here are the details of the discussion and decisions made.
	
They are not Dev Functional's ideas, so when I attack them I mean
no disrespect to Dev Functional.

	1)  It was pointed out that while each language will have merits
	    and strength, however, together they would have both interop
	    (performance) and maintenance issues.

*ANYTHING* is going to have maintenance issues.
And if a system has multiple components, it is certain to have
interoperability issues (which are not the same as performance issues).
And any system is going to have performance issues.

The point is that all of this is meaningless shouting into the wind
without a serious cost estimation.  And the costs of using C++ are far
higher than most people realise.

One of my favourite little books is

	Taming the Tiger:
	Software Engineering and Software Economics
	by Leon S. Levy
	Springer, 1987

(I see Amazon have a used copy for $0.99, so this would be a good time
to buy it.)

Something I got from that book, although not in Levy's own words,
is that a programming language can reduce your costs in three
different ways:

    (A) it can make it easier for you to write what you DO want
	[reducing the SLOC count]

    (B) it can make it harder for you to write what you DON'T want
	[reducing the bugs-per-SLOC count]

    (C) it can make connections in the code more obvious
	[reducing the per-SLOC maintenance cost]

Now most people get fixated on (A).  This is the C++ tarpit.
C++ seems to ignore (B) completely; implicit conversions make the
type system far less helpful for bug prevention than it might be.

Erlang addresses (A):  if you have stuff to do that is naturally
expressed in terms of communicating processes, Erlang makes it dead
simple.  Erlang addresses (B):  "wrong size of integer" errors are
IMPOSSIBLE because there is only one size, "big enough"; treating a
number as an address is IMPOSSIBLE; freeing storage twice is IMPOSSIBLE;
&c &c.  

One way to tell experienced programmers from inexperienced ones (including
ones who have had the same experience repeatedly) is that when you try to
tell them that a language can help you by NOT being able to do things, the
inexperienced programmers cannot understand what you mean, while the
experienced ones say "everyone knows that, but I'm not allowed to use
such a language."

My point here is that the a priori argument for maintenance costs being
far LESS if you use some mix of Prolog, Erlang, and Lisp than in C++ is
really quite strong.  C++ is *too* powerful with respect to the vast range
of low level bugs that are amazingly easy to write, and too weak in other
ways.

That too is shouting into the wind.  Real costing, with real numbers,
is a good idea.  (And real numbers for Lisp, Prolog, and Erlang do exist.)

	2)  It was stated that it is better to fight with the nuances of
	    one language, rather than drool on the strength of each of
	    them independently and then face licensing issues, interop
	    issues, performance issues, support issues , not to speak of
	    the maintenance.
	
Drooling?  Sounds like prejudice talking there.
When a carpenter says "I have some turning to do, so a lathe would be
more useful than a drill press", he's not drooling, he's being competent.

As noted above, there isn't the slightest scrap of a reason to expect
maintenance costs to be lower in a single language (and it turns out that
you _won't_ be using a single language anyway).

	3)  It was pointed out that today even Ericcsson does not use
	    Erlang for its new product development.  Why ?
	
As I understand it from talking to some of the people who were there,
"because the spectacular success of the Erlang projects was making the
let's-use-C++ managers look too stupid."  OK, so that's a bit too strong,
but the largest *failed* project I ever heard of at Ericsson was a C++
project, not an Erlang project.

More generally, if someone is arguing "it is a good idea to use languages
which are good for specific problem areas rather than one-size-doesn't
fit all", it is simply IRRATIONAL to say "then why doesn't everyone use
your preferred language".  I love Prolog a lot.  I wrote a book about it.
I spent years working on it.  But when I have statistical calculations to
do, I use R, not Prolog.  If I have serious number-crunching to do, I'll
use Fortran 90, not Prolog.  If I ever had to do any embedded programming,
I would use Spark/Ada (that is, the Spark subset of Ada) and use the Spark
verifier to make *sure* it was right.  I would not use Prolog.

	4)  The following products of Ericcsson are written in C++
	     TelORB
	     distributed RAM database, the base for the TSP application
	     server platform
	     TDMA-CDMA HLR
	     GSM-TDMA-CDMA mobility gateway
	     AAA server.
	
This is really the same as (3).  At least one mobility server at another
company has been written in Erlang.  

	5)  It was pointed out that clisp has GPL licensing and no money
	    should be wasted on acquiring software licenses (eg.  Franz)
	
Clisp isn't the only Common Lisp out there, not even the only free one.

I have CMU Common Lisp, which is free.  (No, I don't happen to remember
the licence.)  In a previous message I mentioned Poplog Common Lisp.
On my G3 PowerMac I have an evaluation copy of PowerLisp; I haven't
registered it because I'm not using it for software development.  The
registration fee is USD50.

http://www.lisp.org/table/systems.htm has a list of Common Lisp
implementations.  I note that Macintosh Common Lisp is USD750 for a full
licence.  Sounds like a lot, but if a programmer costs USD60,000/year and
overheads are comparable, then it's less than the cost of a programmer
for 2 days, so if using Lisp saved 2 days of development time (how could
it NOT?) it would pay for itself.

I note that "CMUCL is public domain software; users may do anything with
CMUCL and its sources, including linking it to other object files, and
basing proprietary systems on it.", so there are *NO* GPL licence problems
and *NO* money to be spent on software licences.

There are also no licence fees for Erlang, and no barrier to using it
in products.

I also note the language of prejudice:  money is SPENT acquiring
licences; it is not WASTED unless the purchase turns out to be a bad idea.

	6)  It was pointed out that Prolog, CLISP are themselves
	    interested in C!

Presumably that's meant to be "implemented in C".

As far as Prolog is concerned, I can only speak with authority about
the Prolog systems I have personally worked on:

    DEC-10 Prolog:
	The garbage collector was written in Bliss-10.
	The rest of the run-time system was mostly in Prolog,
	extended with some low-level hacks.
	The compiler was written in Prolog.
	There was no C.

    PDP-11 Prolog:
	An interpreter, almost entirely written in assembly code.

    C Prolog.
	This was an interpreter.  Someone (Luis Damas?) wrote a Prolog
	interpreter for EMAS in IMP.  Fernando Pereira needed a Prolog
	on a VAX/UNIX system in a hurry, so translated EMAS Prolog to C.
	Written for portability, not speed.

    Quintus Prolog.
	Written in a mixture of languages.
	The compiler was written entirely in Prolog.
	The user interface was written in Emacs Lisp.
	The garbage collector, block storage manager, and the
	operating system interface were written in C.
	Most of the run-time library was in Prolog.
	The virtual machine emulator was written in Progol,
	somewhere between PL/370 and a macro assembler.
	Note:  C was used for interfacing to things that already had
	C interfaces, notably the operating system.

    Xerox Quintus Prolog.
	Written in a mixture of languages.
	The compiler and most of the run time library were in Prolog.
	The user interface and the rest of the run time library
	were written in Interlisp-D.
	The virtual machine emulator was mostly written in D-machine
	microcode, with Lisp code to back it up.
	There was no C; the machine didn't *have* a C compiler.

    SWI Prolog.
	Written in a mix of C and Prolog.
	Performance is not very good *BECAUSE* so much of it is in C.

I have seen a Prolog implementation for Lisp Machines, and that was
in Prolog, Lisp, and microcode.  But I haven't worked on it myself.

What's the common thread here?  It is that Prolog implementations that
use C use it as a portably asembly/glue language, and that the high
performance ones do NOT have have a VM emulator coded in C (these days
they generate native code, like Aquarius did).

	So, if there is a smart idea present eg.  backtracking, we can
	see the implementation and use it in the code.

This is sheer lunacy.  Have they *any* idea how *HARD* that is?
Do they even have any idea of how little of an idea they have?

	There is no need to take the complete - load of the
	language, since we don't want all the features any way.
	
This is like saying "you tell me that sex with a woman is good.
Well, we don't need no women, we'll just stick the good bits onto a toad."
The problem is that a language like Prolog just doesn't divide up into
bits like that.

	7)  RPC and message passing related weakness in Erlang can cause
	    security vulnerability issues.
	
What weaknesses?  What weaknesses does Erlang have that C or C++ does not?
(Oh wait, there ISN'T any RPC or message passing in C or C++!)

	8)  High performance in computation, high responsiveness in
	    network i/o, aync and disk i/o is mandatory.

This has already been discussed.

	    The performance slide related to yaws was mentioned and the
	    management wondered why the world hasn't moved on to yaws by
	    ditching oh-so-slow Apache ?
	
They really don't have a clue, do that?
Suppose you have built a web site around PHP, which Apache supports
and Yaws doesn't.
Are you going to switch to Yaws?

Anybody who has static pages to serve and nothing else can easily switch.
Anybody who is developing a web site from scratch can easily switch the
design to Yaws.
But someone who HAS a web site with all sorts of SSP and JSP and PHP
and XYZZY stuff that's supported by Apache and not Yaws is going to
find it much easier to buy more hardware than redesign their site.

	8)  The management has decided to organize 4 week long formal
	    rigorous training in C++ and Template Programming.

Do they even *know* that Template Programming *is* functional programming?
(Functional programming in a seriously weird and seriously nasty language,
but not *quite* as bad as XSLT, on second thoughts, since implementations
of the template language are allowed to cut off function calls at different
depths I was wrong, it's much nastier than XSLT.)

I am sorry, but 4 weeks just is not going to be enough.

	There will be sessions in competency building in tools like
	gprof, gdb, g++4, gcc4 etc.

That's nice.  There's two weeks gone.  What are the other two weeks
going to be?

	9)  The decision has been taken by the management and the tools to be used are 
	     - C++ (user land), g++4
	     - C     (kernel land) gcc4
	     - STL
	     - Boost Library
	      There is no license fees to be paid to anybody.
	
And there wouldn't have been had you gone with Erlang and CMU Common Lisp
either.

I am rather worried about "kernel land".  Why are you planning to mess
with the operating system kernel?  (Given that your prototype didn't.)
Do they have *any* idea how much extra knowledge and skill that requires?

	10) All storage related standards have types defined in C and it
	    makes sense to have the programmatic representation as close as
	    possible to the standard.
	
You *can* define storage types in Ada, but you *can't* in C++ or C.
I for one have run into trouble exchanging binary data between two
programs on the *same* machine compiled from the *same* sources;
they had been compiled by different C compilers.

	11) The management also highlighted that if the languages so far
	    used (prolog,lisp,erlang) were so great then the capitalists
	    would have grabbed them up long time ago and made lots of money
	    by now by developing and selling products.  So, either the
	    capitalists are fools or the current development team !

There is so much stupidity in this that it is hard to know where to begin,
and I have to leave in a few minutes.  Basically, management are saying

	You programmers are SO *DUMB* that you can never have any good
	ideas or even recognise any good ideas; the only way we can tell
	if something is a good idea is if everyone BUT you is already
	doing it.

Why do you want to work for these people?

When I heard how much the Bluetail people made out of Erlang,
I felt a very complicated emotion.  As near as I can decode it,
the feeling was
	"I am glad they got so much money, it really vindicates them."
	"I am sad I didn't get any of it."
	"Mind you, they really really deserved it."
	"Why didn't I ask to join them back when I thought they were
	 a bunch of nice, smart, but struggling people?"
	"D--- but they're smart."

Joe Armstrong has a number of articles.  One of them is called something
like "Nobody likes the bearer of good news."

There are (decimal) *order of magnitude* size and cost savings to be
made by using unpopular languages.  To use an unpopular language, you
have to start by admitting that you might be wrong.

Capitalists don't have to make the *best* choices, only better choices
than their competitors.  As long as your competitors are bad enough,
you don't have to be very good to win.

	       It was also pointed out that Google does lot of work
	related to information retrieval, inference, distributed
	computing over thousands of compute nodes but does not use
	Erlang! Why?
	
How do you know Google don't?
How do _they_ know what Google write programs in?

I teach an IR course.  IR is mostly dominated by I/O.  C I/O has very
low overheads.  It's news to me that Google do any serious inference.