new syntax - a provocation

[Joe Armstrong]

On Tue, 23 Sep 2003, Chris Pressey wrote:

> > 	<< The *easiest* way to do "Atom table GC" is not to have an
> > 	atom
> > 	   table in the first place - redesign Erlang so that it does
> > 	   not need an atom table - this is "relatively" easy >>
> 
> Yes, but can you still ensure constant time operations on atoms?

There are two ways of doing this:

Method 1
========

This method assumes the atom table works like a cache and can be
destroyed at any time. If it is destroyed then it is rebuilt
the next time you refer to an atom. This method was suggest to
me by Per Brand at Sics.


Suppose an atom (on the stack or heap) can be in one of two forms:

Originally like this:

	+---------------------+
	| Atom1 | Mod | Index |
        +---------------------+

Atom1 is an "Atom1" tag - Mod and Index are just integer indices
(not pointers).

Now assume there is a global atom table (a hash table).

The first time this atom is referred to, it is changed to

	+---------------------+
	| Atom2 |   Pointer   |
	+---------------------+

ie just like things are today.

Suppose now that the compiler generates code like this:

	pushAtom1
	{atom1, Mod, Index}
	free-space

(these are instructions for a 32-bit threaded machine)

Then the first time this instruction is executed the net result
is to change the instruction to:

	pushAtom2
	{atom1, Mod, Index}
	{atom2, Pointer}

thereafter the value in the atom table is used.

To "garb" the atom table you do a few things:
	
	- junk the atom table
	- scan the code changing pushAtom2 instructions back to
pushAtom1.
	- make a new empty hash table

Every time you junk the atom table a new one will be automatically rebuilt
the next time and atom is referenced. There are a few housekeeping details
to think about - but the method seems sound

Method2
=======

No global atom table.

Atoms in any module are represented

	+---------------------+
	| Atom1 | Mod | Index |
        +---------------------+

Comparison of atoms in the same module is done in unit time.

Comparison of atoms in different modules involves a single string
comparison when the the atom is pattern matched in an exported function call.

If the comparison of


	+-----------------------+
   X = 	| Atom1 | Mod1 | Index1 |
        +-----------------------+
 
and


	+-----------------------+
   Y = 	| Atom1 | Mod2 | Index2 |
        +-----------------------+

  succeeds then we  can always interchange any use of X  with Y or vice
versa. Hopefully choosing the "correct" type for future processing with a
module ...




> > 	- Increase the "isolation" properties of processes
> > 
> > 	<< I want better inter-process protection. One process should
> > 	not
> > 	   be able to crash another though a side effect, like
> > 	   flooding the process with messages.>>
> 
> Also, a process should always be able to tell the pid of the process
> that *really* sent the message.  (Not just any old pid the sender felt
> like sticking in the message.)
> 
> As for flooding, you could, in today's Erlang, write a 'safe_send'
> function that checks the receiver's buffer before sending, but that
> would impact performance negatively, I imagine.  Also - what is the
> desired behaviour for when the receiver's buffer is full?  Block, or
> discard?  The syntax for sending might outgrow nice simple A ! B.
> 
> > 	- Integrate Erlang I/O system with my ideas on UBF - ie
> > 	  make strong contractual obligations on the kind of
> > 	  session dialogs that one Erlang system can have with another.
> 
> Hmmm... I did like the notion of adding state to the type system.  Some
> research into how to create a type system that works well with messaging
> is probably warranted.
> 
> > 	IMHO we need to modify Erlang to increase the isolation
> > 	properties.
> 
> Erlang already has some of the best isolation properties of any
> language, though... the only way I can think to improve them (beyond
> what I've already mentioned) is to make locking easier.  But locking is
> already easy!  So do you have any other concrete ideas on what would
> need to be added/removed/fixed to make processes even better isolated?
> 

Interesting - I'd *much* prefer a solution based on implicit contracts and no
locking - locking distributed objects makes me shudder in horror ..

Assume that there is an implicit contract between any pair of processes of the
form.

	" I will not send you more than 500 messages/sec "

and 
	" I will not send you more than 10MB/sec "

A process violating the sender side of a contract could be punished
with an exit(Pid, youSentTooManyMessages).

At the outer level of abstraction processes are black boxes. Their
communication is regulated by contracts. Contracts can specify
functional behaviour (types) and non-functional behaviour
(example, I will reply within 10ms, I will send max 100 messages/hour).

This is why I'd like to tightly integrate the UBF stuff into Erlang, and
provide wrappers so that components can be written in any language.

> > 	The "make it slower" track has been ignored.
> 
> For better or worse, speed does rule the industry.

  I  disagree -  return on  investment and  time to  market  rules the
industry.

  In my interpretation the  current range of Nortel products developed
in Erlang are successfully because the time to market is very short.

  It takes longer to develop in C - the product is delivered later and
you loose market share during the development period.

  If I  was a venture  capitalist Id go  for early time to  market and
high quality.

  Not fast and late.

  Microsoft  originally gained  market share  by delivering  early and
buggy - now we can deliver early and with very few bugs.

  All that is needed to overthrow the current hegemony is a consistent
growth rate that is higher than the opposition.

> But I grant you that MIPS is not always the best "kind" of speed.
> "Speed" of maintenance is probably the critical thing in my mind.
> Consider: if you don't put the work into making the language
> implementation fast, programmers are going to write their own code in an
> optimized manner to compensate - making it much harder to maintain.


But but ... at home I have my good 'od work horse a 300 MHz Celeron -
is is "fast enough" for everything *except* video rendering.

I have *never* rewritten an Erlang program because it was too slow.

  I have (sometimes) written an Erlang program that was too slow - but
in these cases a bit more  thought convinced me that no language would
help  -  ie  my  algorithms  were intrinsically  slow  no matter  how  I
implemented them.


> > > So please, if we want Erlang to spread, let's keep the language
> > > stable and have a moratorium on new constructs, new bif's etc.
> > 
> > 	No - do the right thing (TM)
> 
> ...and design a new language :)
> 
> -Chris
>