[erlang-questions] Re: No, really, please stop misinterpreting what I wrote

[Michael Turner]

On 9/17/2009, "Ulf Wiger" <ulf.wiger@REDACTED> wrote:

>Some posts refered to the "built-in compression" available
>in Erlang in the form of term_to_binary(Term, [compressed]).
>
>I think one could say that compressing using term_to_binary/2
>is at the far end of the spectrum in this matter. If you
>compress to a binary, you lose every possiblity of pattern
>matching (except check for equality), ...

That's funny.  I just unzipped a file, looked at it in an editor, and
did a search.

What you apparently mean is "you lose every possibility of pattern
matching DIRECTLY on the compressed representation".  But if you
selectively decompress parts of the long list, with some global
dictionary for the more repeated parts, clearly one can do pattern
matching on it.  Will performance necessarily be poorer?  It depends on
the data and its access patterns.  If you can't characterize those, any
optimization attempt will be a shot in the dark anyway.

>This [case of QuickCheck] case in itself answers the 'so what'.
>If it had been feasible to do it in Erlang, the authors
>of QuickCheck surely would have done it by now. Also,
>I would not dream to suggest that they should simply
>choose another data structure in order to solve the
>problem. Their knowledge in that area far exceeds mine.

Well, I guess I wouldn't dream of asking them to choose another data
structure either.  But maybe I would dream of asking them to choose a
different *process* structure.  If passing a rules-base around causes
explosions, how about ... well, NOT passing a rules-base around?  How
about moderating access to the rules-base via processes and messages? 
After all, even if that incurs a performance penalty, it's still a
testing framework, and very likely the code under test will be soaking
up most of the cycles.  Maybe they didn't do this because that's not
how they originally designed the whole thing, and they just didn't want
to bother redesigning it to be more loosely coupled in that respect?

>I've also had discussions with experienced Haskell and
>OCaml programmers who felt that the loss of sharing when
>sending data from one node to another was sufficient
>reason in itself not to use Erlang - since the use of
>sharing in functional programming is such a powerful tool.
>Obviously, I don't share the view that this disqualifies
>Erlang entirely, but at least I'm not alone in thinking
>that (a) sharing is a good thing, and (b) the occasional
>loss of sharing, partly beyond the programmer's control,
>is a bad thing.

Again, I'm new to this, but my understanding of Erlang is that it's not
a functional language per se -- it's a concurrency-oriented language in
which processes are specified in functional terms.  If you're not used
to that emphasis, you won't consider concurrency at every turn, and you
might have accumulated design and coding habits that don't match the
semantic emphasis of Erlang, even if you come from a functional
programming background.

In any case, Erlang's problem as a small minority language isn't to
please the users of other small minority languages so much that it gains
followers from those camps.  It's how to make itself understood -- and
increasingly used for its strengths -- in a much larger software
community, while also becoming more palatable in the process, where it
can do it without compromise of its principle.  If I had to choose
between better syntax and semantics for records and better support for
structure sharing, I'd *definitely* ask for better support for records,
at this point.

>> I assume we'd all prefer everything to be fast, all the time.  That
>> seems to be a preference at Ericsson.  In the telephony switch at
>> Ericsson that has the most Erlang code in it (AFAIK), there's a lot of
>> Erlang code.  But as I understand it, also a lot of C code in that same
>> switch.  They don't use Erlang in that system for speed.  They use it
>> for robustness and expressive power for the concurrency-oriented aspect
>> of the system.
>
>OT, but this is not a correct description.
>
>I assume you refer to the AXD 301? Most of the C code in the AXD 301
>is low-level device processor code, written in C mainly for historical
>reasons. In the first versions, those device processors did not have
>the CPU or memory capacity to run Erlang, and instead ran VRTX - later
>OSE Delta.
[snip]

What it seems to come down to is: Erlang would have overloaded much of
the original hardware, if it would fit at all, so C/real-time-OS was the
better solution at one point.  Later, these components came to closely
match my description of "black boxes you can't get into and fix", for
a variety of reasons including the prohibitive cost of doing that. 
Except for cases like these ...

>There were other blocks of C code, in the form of 3rd party
>applications that were integrated rather than writing everything
>from scratch.....

which were designed in for other reasons that make them virtual black
boxes that you can't get inside of and fix.

Believe me, I spent some time in the trenches in telecom, I know
first-hand how the spaghetti pile grows.

All of which seems to be drifting away from a point: sure, it's nice to
have things arranged so that you can express things naturally in a
language and it will be guaranteed to be fast, or save memory, or have
some other nice feature.  But there's often a hidden or delayed cost. 
Remote procedure calls, for example, seemed like the natural
"idiomatic" way to do interprocess communication.  *Seemed*.  For a
long time.  And to some very smart people, too.

-michael turner