[erlang-questions] cost of integrating 3rd party SW

[Joe Armstrong]

On Sat, May 24, 2008 at 2:55 PM, Ulf Wiger <ulf@REDACTED> wrote:
> Perhaps somewhat related to the "RPC is bad" thread,
> do any of you know of any reasonably systematic
> discussions about whether or not Erlang is good or
> bad at interfacing with 3rd party components?
>
> It's something that pops up every once in a while,
> e.g. in the Facebook chat server article: that combining
> Erlang with components written in other languages is
> hard.

It depends on the non-functional requirements, you must ask questions like:

    - Do the components run on the same machine?
    - Is a bug in one component allowed to crash the other component
    - can we upgrade the components without interrupting the service?

- interfacing *any* two languages hard if you require  things like the above

The only way to do this is to isolate the two components in OS processes
and let them communicate through some kind of stream interface.

This implies that both sides of the stream can easily serialise and parse the
data that is to be transferred. Once you've done this, connecting components is
trivial.

The *easiest* way to connect components is exemplified in the unix shell

cat <foo | grep ... | awk ...| sort | ....

connects components together with minimal effort (and you've written a
parallel program :-)

Of course, linking components into the same address space is more or
less easy if they are written in languages
whose view of the stack and heap is similar - but will cause errors to
propagate between the parts.

If you want to build systems that don't crash then you should isolate
the components in processes -
when you do this the difficultly will be proportional to the semantic
mismatch between the languages.

Erlang has, for example bignums, so connection through a socket to
another language with bignums
would be easy - but connecting to a language with fixnums would be more painful.

IDLs and interface generators like thrift (recently mentioning in the
facebook/Erlang work) help here.
If you use something like thrift I guess there is no difference in
complexity interfacing Erlang to any other
language.


> I know that sometimes, decisions have been made to
> rather write a component from scratch in Erlang, rather
> than interfacing with some COTS component, based on
> the assumption that it will be both cheaper and better
> to do this than to deal with the 3rd party component.

Most often the rewrite is due to the fact that the non-functional
behaviour in the
3-rd part component is wrong. Suppose we want to upgrade things "on the fly"
but this is not possible in the component.

Erlang was designed for building fault-tolerant systems - many components
cannot be just added off-the-shelf because they are less reliable than
the Erlang core.

> This is always a hard sell, and usually meets with the
> exclamation: "aha! Erlang is bad at interfacing with other
> software."

To which you reply "rubbish" - how easy is it to connect a prolog application
running on mac OS-X in Australia to a c# app on .net in China?

>
> And of course, any programmer will always opt to write
> something from scratch rather than reuse something
> that someone else has written, right?  ;-)

Guilty m'lord - but we don't rewrite - we improve, make better versions
that are more easily maintained. 80% of life-cyle costs are in maintenance
so regular re-writes which reduce code volume and improve quality are
justifiable. Those companies who do *not* make these changes
will have mounting problems in the future.

Rewriting assembler apps in fortran was probably a good idea
etc ...
...

> What I'm after is a more sober and objective account,
> based on real experience and (ideally) cost comparisons.
> I'm perfectly ok if the conclusion is that Erlang indeed
> does need to improve.



>
> Cost is of course not the only parameter. Performance,
> robustness, debugging and maintenance, impedance
> mismatch (and if so, what is the preferred solution?
> Keep Erlang or throw it out?)

Keep it

/Joe Armstrong

>
> BR,
> Ulf W
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>