(Prolog + LISP + Erlang) with integration issues versus C++

[Richard A. O'Keefe]

Dev Functional <devfunctional@REDACTED> wrote:
	Given the insistence of the members of the team to use the
	best language for specific purpose, we have code that is
	written in
	 1. Prolog (enumerating device drivers, goal seek)
	 2. LISP   (classical AI stuff, inference)
	 3. Erlang (server, communication and distributedness)
	
Sniff.  Lisp is good for classical AI stuff, but so is Prolog.

	The products we have used are
	 1. gprolog
	 2. clisp
	 3. Erlang/OTP
	
With respect to integration, things *might* have been easier using
either Peter Norvig's "Prolog in Lisp syntax" -> Lisp compiler (as
described in his classic AI book) or the Poplog system.  The Poplog
system integrates
    - Prolog (not quite standard, but certainly close enough to be
              useful)
    - Common Lisp
    - Pop-11 (a member of the Pop family of British AI languages,
      sort of like Lisp in a Pascalish syntax, but it got a number
      of features long before Lisp did)
    - Standard ML
    - dynamically loaded C or Fortran code (I believe it was used once
      in a radar system)
    - a smart screen editor which is, ah, differen, but you don't have
      to use it.
    - a large help system
Poplog used to be sold by Integrated Solutions Ltd.  It has been used
in a number of important products.  For example the SPARK verifier for
Ada is written in Poplog.  These days, however, Poplog is Open Source.

This doesn't do anything about Erlang.  (Given that there is or was
an Erlang-to-Scheme project and that Scheme is not _that_ different
from Common Lisp, it strikes me that it should be possible to develop
an Erlang environment for Poplog.)

While Clisp is a nice implementation of Common Lisp, there are others.
CMU Common Lisp is free and fast.


	The independent modules have been developed successfully,
	however we are unable to surmount the language boundaries
	and connect Erlang to Prolog, Erlang to CLisp and CLisp to Prolog.
	
I must say that I find that ASTONISHING.
Erlang is designed for connecting to things (via the net, of course).
The 2000 version of Clisp certainly supports sockets;
CMU Common Lisp is said to have good foreign language support although
I haven't used it.
Most Prologs these days have some sort of foreign function interface,
some (like Quintus and CWI) have two kinds.  Certainly SWI Prolog and
Ciao have reasonably good foreign function support including sockets.

The whole idea of component-based programming (using things like CORBA,
COM+, &c) is to build software systems out of components connected
through a (possibly local) network WITHOUT having any reason to care
how each component is implemented.

Define the interfaces between your components using UBF, implement UBF
in Lisp and Prolog -- not that hard -- and you are away laughing, no?

	The management is worried about our choice of three languages
	and has strongly suggested a relook at this strategy.
	
You probably have Makefiles and shell scripts as well, which makes five.

	The External Consultant (Subject Matter Expert) has suggested that -
	0. since the individual components are implemented,
	   the idea is partially proven

	1. the language barriers will cause problems in integration
	   and performance

MAYBE.  Maybe not.  The question is HOW MUCH integration do you actually
need?  Where, *exactly* are the performance bottlenecks?

	2. the long-term success and reach of the product will be
	   based on complete implementation in C++.

WHY?  Why C++ rather than C# or Java?

	3. The Visualizer will be implemented in OpenGL and C++ based modules
	   will be quite easy to integrate.

There seems to be a non-sequitur there.  Again, it's a question of how
much integration you need and where the performance bottlenecks are.
If you have clear boundaries between components with relatively modest
amounts of data crossing the boundaries, it shouldn't be any easier to
integrate C++ than anything else.  (Possibly harder:  I have known people
have terrible trouble trying to integrate C++ systems.  One major
European company had such horrible trouble with C++ that they gave up
and implemented their own OO language instead.)

	4. C++ performance will be good if a ANN component is added later
	   due to customer request.

Again, if you treat this as a COMPONENT-based system with clear boundaries
between components, the fact that you have Prolog, Lisp, and Erlang in
there should NOT make it any harder to integrate a C++ component.  In fact
C++ performance with neural nets and other numerical stuff has often been
very disappointing.  To get high performance out of C++ you need to be a
wizard at template metaprogramming -- which should mean high pay and therefore
high maintenance costs -- and to have an extremely good compiler.  But the
big trouble with C++ is that C++ compilers still are nowhere near as
compatible as one would wish.

	5. Multi-language maintenance costs will be high
	
Why?

	Greatly appreciate if the more experience members of this group
	share their experiences and put forward the suggestions.
	
Let me cite R as an example.  It's a statistics environment, originally
developed at the University of Auckland, New Zealand, as an open source
version of the S programming language developed at AT&T.  It contains
code written in

    - C
    - C++
    - Fortran 77
    - Fortran 90
    - R

with the usual shell scripts and Makefiles, plus documentation tools.
When I hear about a new package in the R mailing list, I do

    install.package("the-new-package")

and the R system goes over the web to the R web site, consults the
catalogue to find where the package is, goes over to that site to
fetch it, unpacks it, runs whatever compilers are necessary, builds
the documentation, puts everything in the right place, and cleans up
after itself.  THAT'S integration.  It all just works.

In fact maintenance costs are REDUCED because code already written
doesn't have to be REwritten in order to be used in R.

I could mention Quintus Prolog, which was implemented using
    - Progol (a high level macro assembler, basically)
    - Prolog
    - C (on Unix boxes)
    - Interlisp (on Xerox D-machines)
    - Emacs Lisp
plus the usual Makefiles and shell scripts.  It would have been
*impossible* to implement it all in one language.

	I need to present a case for the proposed solution, on the coming
	Monday.
	
The argument goes something like this:

    - distributed component-based systems are the wave of the future

    - component-based systems are integrated by communicating through
      published interfaces over a (possibly virtual) network

    - this kind of integration does have performance costs (marshalling
      and unmarshalling) but it has reliability benefits (one failing
      component CANNOT corrupt the data structures of another)

    - using more than one language may increase maintenance costs (although
      this is not proven) BUT using the right language for the job may
      dramatically decrease the amount of code that needs to be maintained,
      more than compensating

    - stopping to rewrite things in a language (C++) that the team (who
      you say have a C/Java background) do not know will delay the
      project and have a high chance of introducing new errors.

    - the languages you have chosen all have automatic storage management,
      while C++ does not.  C++ techniques for semi-automatic storage
      management ("smart pointers") have high overheads.  Storage management
      is likely to be a major issue for projects with complex data
      structures, such as this one.  (Yes, there are garbage collectors
      that can be used with C++, but they aren't actually guaranteed to
      be safe.)