new syntax - a provocation

[Joe Armstrong]

On 23 Sep 2003, Mike Williams wrote:

> The best way to "kill" a language is to make continuous additions.
> If people believe a language is in the process of change, they 
> won't use it. The problem is not the additions per se, it is that fact
> that it is very hard to remove older, often not so well thought out,
> constructs. For example I would very much like to see "catch/throw"
> removed, I would like to remove "records" and replace them with some other
> construct. But this would break a huge amount of existing code.
> Every addition will make Erlang biger and more cumbersome to learn and
> use.

Yes -  but ...

We got a lot of things wrong in Erlang and a lot of things right - changing
the language because we got things wrong is IMHO a good idea.

Now a few comments:

	1) Changes should add power to the language *and* make it simpler
	we want fewer constructs that are more powerful NOT lots of special 
	cases.

	My "wish list" has several things (and not of them 
	are changes to guards :-) - Here is a brief list and explanation.
	Stuff in <<...>> are comments (possibly not well thought out)

	- Higher order code

	<< This is "thinking out loud - ..." 
	   something like

	   I think a module should be defined thus:

	   SrcMod = {{export,
		    [{foo,2,fun(I,J) -> ... end},
		     {bar,1,fun(I) -> ... end},
	            ]},
		  {private,
		    [{boo,3,fun(I,J,K) -> ...]}}},
	   Mod = compile(SrcMod),
	   Mod:foo(1,2) ...


	   install(Mod, quack)
	   quack:foo(1,2)

	   Src1 = replace(foo,2,Mod, fun(I,J) -> ...)
	   ...

	   Then I'd like not TWO but an indefinite number of "old
	   versions" of the code - let the GC remove code that is 
	   not reachable.

        >>


  	- Proper structs

	<< junk the record stuff. thus has been discussed in previous
	   mails >>

	- remove the atom table.

	<< The *easiest* way to do "Atom table GC" is not to have an atom
	   table in the first place - redesign Erlang so that it does not
	   need an atom table - this is "relatively" easy >>

	- Increase the "isolation" properties of processes

	<< I want better inter-process protection. One process should not
	   be able to crash another though a side effect, like
	   flooding the process with messages.>>

	- Integrate Erlang I/O system with my ideas on UBF - ie
	  make strong contractual obligations on the kind of
	  session dialogs that one Erlang system can have with another.


	2) We have to distinguish the *essential* features of Erlang from
	the non-essential.

	IMHO the *essential* features are:

	- light weight concurrency
	- inter-process isolation
	- transparent distribution
	- remote detection of errors

	All the rest (HOFs, structs, List comprehensions, type systems,
	bit syntax, macros, includes, ... bla bla) are "nice to have"
	but NOT essential.

	IMHO the most essential of the essential features is "isolation".

	Operating system processes do two things - they allow transparent 
	resource sharing (like CPU time slicing) and protection.

	One process cannot muck around with another processes. To me this
	is the *essential* feature of a processes. It allows us to protect
	one body of code from another - it allows groups of people to
	write code. Without this protection any error in any code could crash
	the entire system.

	Since most languages (java, C++, C, .....) do not have any notion
	of protection they cannot be used for any large bodies of code in a
	reliable manner (well they *can* be - but it is very difficult).

        Process provide "protection domains" - unfortunately OS processes
	are big and heavy (they don't have to be - <<in some (rare) Os's
	there are light-weight processes>> but in the dominant unix/windoze OSs
	processes are heavy>>

	Because processes are big and heavy, the notion of "threads" emerged
        (threads share resources) and are lighter weight than processes -
	but oops - out went the baby with the bath water. We lost protection -
	wow - great - fantastic - super - wooppee - not we can write
	super duper fast programs which crash the system - joy.

        Enter Erlang/Oz, ... whose processes are lighter-weight than
	threads AND have the protection (but not quite - I suspect
	that OS processes are better protected than Erlang processes).

	IMHO we need to modify Erlang to increase the isolation properties.


	3) On language implementations.

 	 All  these  nice  things that  people  want  are  a *pain*  for  the
         implementor. A small number of very general things is easier to 
	 implement than a lot of special cases.

	The last ten years of Erlang development have followed the
        "make it faster"  track - ie keep the language the  same but make it
        faster.
	
	The "make it slower" track has been ignored.

	Where is the "pessamizing compiler?" - Personally I am interested in
	the "very small, very easy to understand - very easy to port" -
	very well documented" type of implementation.

  	I have been re-working the system with this in mind.

	Here all the things that make the programmer's life easy make
        my life a pain.

        Erlang variable scoping rules are, for example nice for the user
	but a pain for the compiler writer.

	As regards a simple implementation I am of two minds here -
	I have a very nasty case of "can't make my mind up" ism - here
	are two approaches:

	A) Transform to a kernel language and then do an
	efficient (yes I said it :-) compilation of the kernel language.
	<<ie transform out cases, if, func, lc, records into kernel Erlang
	and compile what's left>>

	B) Compile everything in-place.
	ie let the kernel compiler know about cases, funs etc.

	So far I'm doing B) - but the B approach involves a virtual machine
	which has specialized instruction for handling closures,
	list comprehension iterators etc - the nice language features
	that we all want to have can be either rather easy or very difficult
	to implement at this level.

	I guess A) might be better - but would it be sufficiently 
        efficient - compiling out  cases, ifs etc. lambda-lifting, and "list
        comprehension lifting"
	(is their such an expression) results in many extra function calls
	and a lot of additional argument shuffeling on the stack or in 
        registers.

	I guess the real answer is "do it both ways and measure" ... :-(

	<<In any case I think a code-to-kernel transformer is pretty
	  useful ... - and yes I know there *is* a kernel form -
	  but even it is is rather too complex for my taste - I'm
	  thinking more of the pure lambda calculus augmented with
	  processes >>
	
	

> 
> So please, if we want Erlang to spread, let's keep the language stable
> and have a moratorium on new constructs, new bif's etc.

	No - do the right thing (TM)

	I think the right thing is to *remove* non-essential stuff and
build on the central core concepts -

	Increase the isolation property of processes.

	This makes Erlang more useful.

	/Joe


> 
> /mike
> 
> 
> In article <Law12-OE52BXx4XFKpR00001991@REDACTED>,
>  you write:
> |> Hi,
> |> 
> |> Since new syntax constructs have been discussed, I have some ideas that I'd
> |> like to present here. The intention is to spur some more ideas, and maybe
> |> some will find a growing ground.
> |> 
> |> * I have found myself wishing to be able to build data structures other than
> |> tuples or lists, in the same elegant way as tuples and lists, directly in
> |> the code. The most acute need was for bitsets/integer sets, where using the
> |> binary syntax is very cumbersome when there are many such sets to be
> |> declared. It would be super-nice to be able to write something like
> |>     #[0011010001]
> |> or
> |>     &[3, 45, 27, 39],
> |> where the first is a bit representation, and the latter a set containing the
> |> specified integers. The implementation would be as binaries, so just the
> |> syntax is new.
> |> 
> |> * I thought this could be solved with a parse transform, but I think the
> |> lexer should be able to cope with new tokens too, since overloading existing
> |> ones would only make a mess out of it. That's why I thought "why not
> |> introduce something similar to Lisp's reader macros?" We could have a
> |> special char (for example, @) to use in constructs like
> |>     @[....]
> |> that will be transformed into something like (in this case)
> |>     mylexer:'['(....)
> |> and evaluated at compile time into "real" Erlang.
> |> 
> |> * Having come this far, and remembering Tobbe's (I think) call for Erlang2,
> |> and also Luke's Lisp parallels, there is one more small step to take, for
> |> those that like to play on the edge: Why not replace the whole Erlang syntax
> |> to a more Lispy one? (or more correctly, closer to CoreErlang) A
> |> different-looking language, but the same BEAM code and runtime. There are
> |> many aspects of such a syntax that add plenty of power to Lisp, why not use
> |> that power?
> |> 
> |> Note: I am too aware of some of the (many) downsides. I tried to be a little
> |> provocative by leaving them out, and also any examples and possible
> |> advantages.
> |> 
> |> regards,
> |> Vlad
> |> 
>