[e-lang] Proposal: E / Erlang integration

[Mark S. Miller]

Michael FIG wrote:
 > I am writing because I think that E uses an excellent architecture
 > for sending messages between mutually-untrusting components (based
 > on _promises_, also known as _refs_ or _vows_), and because I like
 > Erlang (it is fun to write applications using a dynamically-typed
 > functional language).


Hi Michael,

I also like Erlang a lot, and I agree that there are many connections
between it and E. Peter van Roy (one of Oz's creators), in
"Convergence in Language Design: A Case of Lightning Striking Four
Times in the Same Place", makes some comparisons between Erlang, E,
and Oz at
<http://research.microsoft.com/~leino/ifip-wg2.3/media/45/VanRoy-FLOPStalkIFIP.ppt>
and <http://www2.info.ucl.ac.be/people/PVR/flopsPVRarticle.pdf>.

Section 24.2 (p. 187) of my thesis, at
<http://www.cypherpunks.to/erights/talks/thesis/markm-thesis.pdf>
says:

 > Joe Armstrong's Making Reliable Distributed Systems in the Presence of
 > Software Errors [Arm03] explains the design and rationale for the
 > Erlang language. Erlang has many similarities with E. Erlang is a
 > dynamically type safe distributed language, inspired by Actors, and
 > based on communicating asynchronous messages between encapsulated
 > processes. It was designed for writing systems that remain reliable
 > despite software errors. Erlang has been in commercial use at Ericsson
 > as the basis for their highly reliable (five nines) telephone
 > switching products. This represents an empirical demonstration of
 > robustness *well* beyond anything we have accomplished. One particular
 > notable accomplishment of Erlang---well beyond anything in E but
 > needed for E to achieve its goals---is Erlang's support for robustly
 > upgrading code in a running system.


I believe the E and Erlang efforts have much to learn from each
other. I have certainly learned much of value from Erlang already, and
have encouraged others to do likewise. I think the questions you and
Bengt Kleberg are asking, of how to embed elements of E's semantics
into Erlang, and what (if anything) this would contribute to Erlang,
are an excellent way to start exploring these issues.



 > In my mind, the only things that would need to be changed in Erlang
 > in order to support E's semantics would be syntax to send E-style
 > messages (that result in a promise), a construct to invoke code when
 > promises are fulfilled, and a flag to indicate that a specific
 > process is untrusted and may only interact with the rest of the
 > system by receiving and sending E-style messages.



I would like to discourage you from starting with syntax. First, try
creating an Erlang library exporting operations that correspond to E's
primitives. Then ask what value (if any) these add to Erlang.

I think you have correctly identified the two areas where E's ideas
may contribute to Erlang: concurrency control and security. More on
these below.


 > The advantage to the E community would be yet another implementation and
 > popularization of E.

We try to be careful to distinguish between two ways to adapt E to
other language platforms. One is to port E onto the other platform
(E-on-Java, E-on-CL, E-on-Squeak, E-on-C), so that existing E programs
(ideally) run compatibly. The other is to adapt the other platform so
that it may benefit from ideas derived from E (Joe-E, Twisted Python,
Squeak-E, Emily, Oz-E, CaPerl, etc). From your message, I gather that
you are not actually proposing "another implementation" of E, but
rather an adaptation of E ideas to Erlang. I think this is a wise
choice. While E-on-Erlang would also be an interesting project, and
one that I'd love to see happen, it would probably teach each
community less about the other.


 > The advantage to the Erlang community would be a robust model for
 > secure distribution and a mechanism for executing untrusted Erlang
 > code which is far more effective than "sandboxing" techniques.
 > Initially, I only want to define the syntax and semantics and
 > implement them in Erlang... adding cryptography and compatibility
 > with the existing Elib is something that I would like to defer until
 > later.

I would drop compatibility as a goal from the project you have in
mind. Again, this would be more appropriate to an E-on-Erlang project,
which would be quite a different effort.


 > I am prepared to work on the implementation, but I do need to hear
 > what people in each community think about this.  I would like to
 > come up with something that both communities are happy with, so that
 > we can work together.

I for one think this is a great idea! Please continue to post on your
progress and any questions you have.



Bengt Kleberg wrote:
 > presumably this only shows my ignorance, but i hope that it will be
 > possible to reduce it after this question :-)
 >
 > why do erlang need a special syntax to send e-style messages? what
 > does this add that the normal send (!) does not have?  why do erlang
 > need a special syntax to receive e-style messages? what does this
 > add that the normal receive does not have?

As I said to Michael, I think any attempt to provide special syntax
should be postponed.

Regarding concurrency control, E's event-loops are non-blocking. By
contrast, the preferred Erlang style for sending a request and
receiving a response is for the requesting process to do a blocking
receive that is receptive only to the alleged response (where the
alleged response is distinguished from other requests by
convention). This would seem to leave networks of Erlang processes
vulnerable to deadlock. Continuing with section 24.2 of my thesis:


 > Although Erlang was inspired by Actors, a common programming pattern
 > in Erlang is for a process to block waiting for a reply to a
 > previous request. While blocked, the process is not responsive to
 > other requests. This is like the "message-oriented systems" of Lauer
 > and Needham [LN79] rather than the non-blocking event loops of E.

 > [LN79] Hugh C. Lauer and Roger M. Needham. On the Duality of
 > Operating System Structures. In Proc. 2nd International Symposium on
 > Operating Systems, pages 3--19. IRIA, October 1979. ACM Operating
 > System Review.

However, if Erlang systems don't seem to be vulnerable to deadlock in
practice, the Erlang community may not find the above difference
compelling. OTOH, I consider the following issue urgent:


 > Although Erlang is not engineered to provide robustness against
 > malice, it seems to be an object-capability language. Erlang
 > processes are encapsulated objects and processIds are unforgeable
 > capabilities, providing the right to send messages to a process. The
 > rules by which processIds are transmitted and acquired seem to be
 > precisely object-capability rules.  Most importantly, the language
 > has no global scope. Each process has its own separate state.

 > Unfortunately, a process has only one processId, so the common
 > convention is for making a request and receiving a reply is for the
 > client to include its own processId in the message as a reply
 > port. This gives the provider the authority, not only to reply to
 > the client, but to invoke the client. It is not clear how to repair
 > this convention within the overall Erlang framework.


This issue seems fundamental. It cannot be fixed either by syntax or
by a library. Until it is fixed, I don't see how to write practical
Erlang programs with distributed object-capability security
properties.

-- 

     Cheers,
     --MarkM