[erlang-questions] Ideas for a new Erlang
Richard A. O'Keefe
ok@REDACTED
Thu Jun 26 05:57:10 CEST 2008
On 25 Jun 2008, at 2:23 am, Sven-Olof Nystr|m wrote:
>
> Hello everyone,
>
> I have written down some thoughts and ideas on the future development
> of Erlang.
I spent last night reading "Ideas for a new Erlang". It's not clear
whether it would be better to write all the responses in one message
or to split them into several messages, each with its own Subject:.
Let's keep it as one to start with.
SELECTIVE RECEIVE VS MAILBOX OBJECTS
====================================
p1. "Consider, for example, selective receive (no other programming
language requires anything similar),"
Untrue. Ada and Occam should not just spring to mind but jump up
and down screaming loudly "pick me, pick ME!"
9.7 Select Statements
[There are four forms of the select_statement.
One form provides a selective wait for one or more
select_alternatives.
Two provide timed and conditional entry calls.
The fourth provides asynchronous transfer of control.]
9.7.1 Selective Accept
[This form of the select_statement allows a combination of waiting for,
and selecting from, one or more alternatives. The selection may
depend
on conditions associated with each alternative of the
selective_accept.
{time-out: See selective_accept} ]
Syntax
selective_accept ::=
select
[guard]
select_alternative
{ or
[guard]
select_alternative }
[ else
sequence_of_statements ]
end select;
guard ::= when condition =>
select_alternative ::=
accept_alternative
| delay_alternative
| terminate_alternative
accept_alternative ::=
accept_statement [sequence_of_statements]
delay_alternative ::=
delay_statement [sequence_of_statements]
terminate_alternative ::= terminate;
A selective_accept shall contain at least one accept_alternative.
In addition, it can contain:
• a terminate_alternative (only one); or
• one or more delay_alternatives; or
• {else part (of a selective_accept)} an else part
(the reserved word else followed by a sequence_of_statements).
These three possibilities are mutually exclusive.
The quoted material is from the Ada'05 (actually ISO something:2007)
reference manual, however the construct was already present in Ada 81.
The placement of the guard *before* the accept tells you something
relevant. In Ada (and Occam), the guards serve only to selectively
enable or disable which messages will be looked for *before* they are
considered for accepting, not to select or postpone messages based on
their content after they arrive as in Erlang. However, nothing in the
Erlang language requires the guards in a 'receive' to depend on the
message contents, so the Erlang selective receive can be used to do the
Ada (and Occam) thing.
The classic example is a bounded buffer.
buffer(Status, Contents) ->
receive
{put,Msg} when Status /= full ->
{Status1, Contents1} = add(Contents, Msg)
; {get,Who} when Status /= empty ->
{Status1, Contents1, Msg} = pop(Contents),
Who ! Msg
end,
buffer(Status1, Contents1).
Without selective receive (or in Ada, selective accept), this would
require three receives: one for the full case, one for the empty case,
and one for the in-between case. In general, this lets us replace up
to 2**n-1 receives with a single receive having n guarded alternatives.
p3. "It is strange that Erlang has a feature that is on one hand so
general and powerful, but yet programmers are not supposed to
take advantage of it!"
It would be strange, IF it were true. But it isn't. Erlang programmers
ARE supposed to take advantage of selective receive. Just as selective
accept is indispensable in Ada and Occam, it is indispensable in Erlang.
There is no other way, in any of these three languages, to wait until
one of N things happens.
Note that "simplifying" Erlang receives would do nothing whatever to
address the problem of mailbox overflow. Supposing Ch to be a channel,
L = lists:seq(1,1000),
lists:foreach(fun (X) ->
lists:foreach(fun (Y) ->
lists:foreach(fun (Z) ->
Ch ! ok
end, L)
end, L)
end, L)
is going to be trouble even if there is only unconditional-receive-from-
channel and no selective receive at all.
It would be absurd to cripple the language in order to avoid a problem
and then find the problem snickering on your shoulder as lively and as
malevolent as ever.
p3. "To the best of my knowledge, selective receive has only two uses:"
The bounded buffer example above shows that this is not so.
p4. Channels as first class objects.
But they already are. There are three proposed functions and an
operator.
We'll represent a channel as a process. The only even slightly tricky
thing is that we need to distinguish between other messages sent to the
current process and replies sent from a channel. We'll use the rather
subtle fact that a reply can only arrive when we've asked for one, and
that when we ask for one, we'll wait until we get it, so there never
will
be one in the mailbox at other times.
current() ->
self().
new_channel() ->
Original_Self = self(),
spawn_link(fun () ->
new_channel_loop(Original_Self)
end).
new_channel_loop(Original_Self) ->
receive Message ->
receive Original_Self ->
Original_Self ! {self(),Message}
end,
end,
new_channel_loop(Original_Self).
ne_receive(Pid) ->
Pid ! Self,
receive {Pid,Message} ->
Message
end.
ne_receive() ->
receive Message ->
Message
end.
So anyone who wants to program in this style already may. In fact it
is useful to distinguish between a single-shot channel and a multi-shot
channel. I hope that we can rely on Erlang garbage collection to
collect any process suspended in a receive construct and not registered
and not referenced by any other process. If we can, the distinction is
only useful to people and not really needed by the computer.
new_reply() ->
Original_self = self(),
spawn_link(fun() ->
receive Message ->
receive Original_Self ->
Original_Self ! {self(),Message}
end,
end,
end).
The example
f(Counter) ->
Counter ! inc,
Counter ! inc,
Q = new_channel(), // create a new channel...
counter ! {value, Q},
Value = ne_receive(Q),
% since when has Erlang had BCPL-style // comments?
should really be
f(Counter) ->
Counter ! inc,
Counter ! inc,
R = new_reply(), % a new single-shot channel
counter ! {value, R},
Value = ne_receive(R),
The big question, as always, is "what does SIMPLE mean?" and the answer
is seldom simple. Nystrom claims that replacing Erlang's
one-mailbox-per-process-with-selective-receive
approach with
processes-plus-channels-with-unconditional-receive
provides "a much simpler mechanism".
The *mechanism* may be simpler, though given the
support for pattern matching that has to be there
anyway, it is not clear to me that it really *would*
be simpler. Whether it is a simpler system to *use*
is another matter entirely.
The thing that strikes me forcibly is that I have seen
all this before. John Reppy's Concurrent ML. It's
supported in SML/NJ and Mlton and perhaps other ML
implementations. What Nystrom calls a "channel" is
what CML calls a "mailbox". (What CML calls a "channel"
is what Occam calls a "channel".) And comparing the CML
code I've seen to Erlang makes Erlang look stunningly
simple *in use*.
GUARDS
======
p6. "The complex selective receive primitive of old Erlang
has forced the Erlang designers to introduce a more
restricted type of expressions called _guards_."
This is fiction, and it's fiction with a major error in it.
Guards are *NOT* a restricted type of expression.
This was much clearer in old Erlang.
Sadly, it has been obscured by the disastrous design bungle
of giving guard predicates aliases that can be used in
expressions and by allowing 'or' and 'and' -- which should
be avoided AT ALL TIMES AND IN ALL PLACES -- in both contexts.
It would be really really useful if there were a compiler
option to complain loudly whenever a guard predicate's alias
is used in an expression.
The *syntax* of guards in Erlang is similar to expressions,
but the *semantics* is very different. Consider:
Guard Expression
succeeds returns 'true'
fails returns 'false'
returns something else
raises an exception
Some forms that would raise an exception as an expression
fail as guards. Some forms that would raise an exception
as an expression succeed as guards. The order of evaluation
of expressions is defined. The order of evaluation of
guards is not defined. If you take a guard and treat it as
an expression, expecting it to always answer 'true' or 'false',
you are in for a nasty surprise:
some guards that fail may raise an exception
some guards that succeed may raise an exception
some guards that succeed may return a value other than 'true'.
The whole point of section 3.4 is
"SINCE guards are restricted expressions,
they ought to be unrestricted expressions"
but since the assumption is flat-out false, the conclusion is
unwarranted.
The limitations of guards have NO intrinsic connection with
receive, although they turn out to be very pleasant for receive.
Erlang guards have their roots in concurrent logic programming
languages such as Strand88, GHC, FCP, and Parlog.
Whenever I see people demanding that guards should be complexified,
I find myself wondering "do I know something about readability and
maintainability that they don't, or is it the other way around?"
The way I see it, if your selection process is complex enough that
you *want* to write complex guards with user defined functions,
then what you *need* is either
- abstract patterns, or
- a rewrite.
Let me offer an example I saw today, in
http://www.infoq.com/articles/erlang-dsl
It doesn't involve a user defined guard, but it does show what
happens when you have a "keep-guards-simple" mindset.
Here's the original code. Its purpose is to return a function
that sends a message if something has a value within a limit.
to_function(Pid, Action, Quantity, Ticker, Operator, Limit) ->
fun (Ticker_, Price) ->
if
Ticker =:= Ticker_ andalso
( ( Price < Limit andalso Operator =:= less ) orelse
( Price > Limit andalso Operator =:= greater )
) ->
Pid ! {Action, Quantity, Ticker} % place an order
; true ->
erlang:display("no rule applied")
end
end.
I looked at that and, well, it didn't make me smile.
That complex guard *had* to go. A few seconds later:
to_function(Pid, Action, Quantity, Ticker, less, Limit) ->
fun (T, Price) when T =:= Ticker, Price < Limit ->
Pid ! {Action, Quantity, Ticker} % place an order
; (_, _) ->
erlang:display("no rule applied")
end;
to_function(Pid, Action, Quantity, Ticker, greater, Limit) ->
fun (T, Price) when T =:= Ticker, Price > Limit ->
Pid ! {Action, Quantity, Ticker} % place an order
; (_, _) ->
erlang:display("no rule applied")
end.
In the original context, this now gives us the benefit of a 'compile
time'
failure if Operator is not 'less' or 'greater', rather than a 'run time'
failure. But it gets better. This is about buying or selling stocks,
and it so happens that we always want to buy when the price is LOW,
and sell when the price is HIGH. So now we can write
to_function(Pid, buy, Quantity, Ticker, less, Limit) ->
fun (T, Price) when T =:= Ticker, Price < Limit
-> Pid ! {buy, Quantity, Ticker}
; (_, _)
-> erlang:display("buy rule did not apply")
end;
to_function(Pid, sell, Quantity, Ticker, greater, Limit) ->
fun (T, Price) when T =:= Ticker, Price > Limit
-> Pid ! {sell, Quantity, Ticker}
; (_, _)
-> erlang:display("sell rule did not apply")
end.
Now we can *see* that the returned function will buy if the price is
low or sell when the price is high, but not vice versa. This was not
possible with the original version.
What I've done here is to trade size against readability. The next
step would be to replace Pid by Broker, but that's a different
readability issue.
LINKED MODULES
==============
p6. "Erlang's record facility is rather unusual (at least in the
world of functional programming languages)."
If you are willing to count the Lisp family as functional, then
this is historically quite untrue. I've lost count of the number
of times I've written Scheme code like
(define (make-node Op Left Right)
(vector 'node Op Left Right))
(define (node? Node)
(and (vector? Node)
(= (vector-length Node) 4)
(eq? (vector-ref Node 0) 'node)))
(define-macro (node-op Node)
`(vector-ref ,Node 1))
(define-macro (node-left Node))
`(vector-ref ,Node 2))
(define-macro (set-node-left! Node New-Left)
`(vector-set! ,Node 2 ,New-Left))
...
The problems of -include have been known for a long time.
My "Delenda est preprocessor" paper was written some time
in July 1998 (at least, that's the date on the only copy
I can find).
p7. "the obvious alternative would be to expand a record definition
into a set of function definitions".
Well, no, because you can't use function calls as patterns.
That's why abstract patterns are needed.
The work I did for Fergus O'Brien back in 1998 -- I miss him,
and I especially miss getting paid for thinking about Erlang! --
included a proposal that I'll now reconstruct for you, with
two small changes.
Two new directives replace -import(Mod, Funs).
-import(Module, When, Funs).
-use(Module, When, Funs).
Module : an atom
When : 'early' or 'late'
Funs : a list of Name/Arity (or #Name/Arity) pairs.
A -use directive creates a dependency (recorded in the .beam
file) between this module and Module. It also notifies the
compiler that this module may call any of the functions (and/or
abstract patterns) listed in Funs, but if so, a module prefix
is required.
If When is 'late', the dependency is a load-time one: before
EXECUTING this module, Module should be LOADED. Cyclic late
dependencies are perfectly OK; all the members of a cycle
should be loaded before running anything from any of them.
This dependency exists even when Funs is empty; you might be
using some kind of dynamic call to call functions from Module.
The only thing that the compiler may assume about Module is that
loading will have ensured that the named functions all exist,
even if only as loader-generated stubs that raise an undefined
function exception.
If When is 'early', the dependency is a compile-time one:
before COMPILING this module, Module should be COMPILED, and
so if Module is changed, this module also needs recompiling.
Cyclic early dependencies are not allowed. The compiler may
assume about Module anything whatever that it discovers in the
.beam file. (Arguably interface information should be in
a separate file, not unlike the .hi files used by GHC.
I propose ".ern" (for .ERlang iNfo), in honour of the late
Ern Malley.)
The difference between -import and -use is that -import
allows (but does not require) the listed functions to be called
without a module prefix, provided there is no clash with any
other imported function or any function defined in this module.
There may be more than one -use or -import for a Module, and
there may be both -import and -use directives for the same
Module. There may be both early and late dependencies on a
module, but if so, 'early' wins.
early/late is split out as a separate argument so that it can
be determined by a macro.
BINDINGS VIA A TYPE SYSTEM
==========================
p8. "If we compare with the equivalent code in ... SML"
But why choose SML as the comparison? Haskell's
y*y where x = 42, y = x+x
requires only one keyword ('where') instead of SML's four.
I'm not arguing for adding 'where' to Erlang, just pointing
out that comparisons against a limited choice of comparands
can be misleading.
As a matter of fact, I rather like this proposal, and think
it might be a nice style check to have. Although it _is_ a
bit dodgy that
X = 1 exports X
X = 1, ok does NOT export X
and it's a bit of a worry that
X = Y = f()
is a type error in the proposed scheme. I found another
problem last night, but it was 1am and I can't read what I
wrote.
I think this proposal should be finished and written up
separately and EEPed as a style check.
A MINI LANGUAGE
===============
The reference to the CELL is a little inaccurate, not
that it matters. (The heart of a CELL is a two-way
CMT version of the PowerPC, with AltiVec.)
This is an interesting idea, but there isn't really a
proposal here yet. It will be very interesting indeed
to discover what Apple's OpenCL looks like; in the
mean time, there's CUDA.
The trick with something like this is that there really
isn't a whole lot of point in it unless you go native.
OpenCL, of course, relies on LLVM.
I suspect that
- some sort of opaque numeric array values (not unlike
APL)
- calling out to high performance libraries like
ATLAS, the CBLAS, GSL, perflib, &c
might be a more effective way to take advantage of
the vector processing facilities in modern machines.
Whoo, wouldn't that be fun! A hybrid of Erlang and APL2!
MACROS
======
p14 "I don't know of any language that combines Lisp-style
macros with a syntax that isn't Lisp-like."
`C (Tick-C), and arguably, Template Haskell.
'forms' in WonderPop (the Pop-2 implementation for TOPS-10
that Robert Rae worked on) are closer to Scheme's macros.
From my experience with Lisp, there are macros that could
have been written using Scheme syntax forms, and there are
macros that make essential use of assignment. Using Nystrom's
own example, you really _can't_ translate a "def-nt" form into
(part of) a LALR parser in isolation; the information has to be
set aside until you have seen all of the rules for that grammar.
That setting aside involves some sort of data base or other
mutable variable.
I note that the closures proposal for Java includes bending
Java syntax so that things like
with_lock (Some_Lock) { statements... }
which are traditionally handled in Lisp using macros
are in fact procedure calls, so that inlining does practically
everything you want. Here's the trick]
[for] Primary (FormalParameters : [Expressions]) Statement
==>
Primary(Expressions, {FormalParameters => Statement});
[for] Primary ([Expressions]) Statement
==>
Primary(Expressons, {=> Statement});
where {formals => statements [expression]} will be Java's syntax
for a lambda-expression.
An Erlang version might look like
'do' [module:]function([patterns||][expressions]) '->'
expressions
'end'
=>
[module:]function('fun'([patterns]) '->'
expressions
'end'[, expressions])
e.g.
do lists:foreach(Toy || Sack) ->
play_with(Toy),
break(Toy),
cry()
end
For Erlang, no big deal. For Java, a giant step.
See http://www.javac.info/closures-v05.html
The big deal about Template Haskell, of course, is both fitting in
with *and* taking advantage of the type system, something which is
not an issue in Erlang.
More information about the erlang-questions
mailing list