A Joeish Erlang distribution (long)

[Kenneth Lundin]

Joe Armstrong wrote:
> On Mon, 27 Jan 2003, Vance Shipley wrote:
> 
> 
>>On Mon, Jan 27, 2003 at 05:32:33PM +0100, Kenneth Lundin wrote:
>>}  
>>}  Here are some comments from the Erlang/OTP Product Manager at Ericsson.
>>}  
>>}  1) We have plans to divide the Erlang/OTP distribution into s number of
>>}  separate packages. Preliminary we are thinking of 3 packages:
>>}  - Core package (erts,kernel,stdlib,sasl,mnesia,os_mon,inets,compiler,..)
>>}  - Tools&Utilities (parse_tools, tools, debugger, observer, et, tv, ...
>>}  - Telecom/protocols (snmp, asn1, orber, megaco, ...
>>
>>I would suggest that the core package be even more limited.  All you really
>>need to run Erlang is erts, kernel and stdlib.  A "Development" package 
>>would be appropriate.  If you think in terms of embedded systems and what
>>they require it makes a cleaner seperation.

I think a "core" package should contain the basic runtime environment,
minimal set of tools for creating your own erlang modules (i.e the 
compiler) and some applications that almost everyone who
want to try Erlang is interested in.

snmp of course out, mnesia is in or out.
> 
> 
>   I  entirely agree  - snmp  mensia etc  are *applications  that are
> written in Erlang - they should not therefore be in the core release.
> 
>   The core should be:
> 
> 	1) The compiler 
> 	2) The run time
> 	3) A minimal set of libraries to do something "useful"
> 
>   Think C - a minimal set of stuff is
> 
> 	- A compiler that runs "out of the box"
> 	- A minimal set of libraries (libc) so you can do something.
> 
>   IMHO we should have:
> 
> 	- A compiler that runs "out of the box"
> 	- Minimal library support
> 	  Somewhere you have to "draw the line" 
> 
> 	My "minimal support" really means device drivers - so support
> for files, sockets, binary filers, ... etc. is in - everything else is out.
> also ets is so low level that it can be considered "part of the language"
> 
>    We have to be pragmatic as well so I'd include
> 
> 	- dets
> 	- dynamic code loading
> 	- the shell
> 
>    As well in the core.
> 
>    (But no snmp, mnesia, .... asn1 etc.) these are *applications* written in
> Erlang - and no gen_server ... etc.)
> 
>   IMHO  gen_tcp  etc.  dets  should  be  written  so  they  don't  use
> gen_severer etc.  - BTW my stand-alone Erlang broke just when I wanted
> to use dets and gen_tcp - the reason was the dets and gen_tcp uses all
> the application stuff and the gen_* stuff.

The division of the Erlang/OTP distribution into several (e.g 3) 
packages and the reorganization of individual applications such as 
kernel, stdlib etc. are two different matters that are to treated 
separately.
I agree that there are many things within kernel and stdlib that 
could/should be rewritten to make the "core" parts cleaner independent 
from other "optional" stuff. BUT the division into several packages will 
not involve rewriting of modules or dratical changes in the contents of 
kernel and stdlib.
Some small justifications are planned, such as a removal of the 
snmp-mibs from sasl. Since several years this unfortunate placing of the 
MIBs has created a build dependency between the current core parts
and SNMP forcing every one to build the SNMP compiler before the
runtime system could be built.
> 
>   One  of the  basic design  principles  underlying the  system is  (as
> Martin Björkland said)
> 
>   "There is stuff  that has to be right, and stuff  that doesn't - you
> have to make you mind up."
> 
>   IMHO the core should NOT be able to recover from errors - you get the core
> right - period.
> 
>   Applications  outside the  core can  if they  wish build  upon gen_*
> modules  to make the  apps. fault-tolerant  - but  the core  should be
> A. Priori assumed correct - and have no such code.
> 
>   This is *why* the core should be as small as possible and include as
> much pure code as possible (i.e code like in lists etc.)
> 
>   I think the discussion should proceed upon a module per. module basis.
> 
>   We should not be talking about which component lives in the core
> (i.e. do we include snmp, corba etc.) but the discussion should be ...
> 
>   The core is compiler+stdlib+sasl+kernel with the following
> changes:
> 
>   The following modules are *removed* 
> 	
> 	supervisor_bridge.erl
> 	sos.erl
> 	supervisor.erl
>      	gen_*.erl
> 	...       etc.

I agree with the way of thinking but the number of modules in kernel and 
stdlib is not a problem for most of our customers, it is mainly a 
problem (small) for us when maintaining and developing the system.
If we make drastical changes in these applications we will whoever 
create problems for our customers. Therefore we will make such changes
step by step in a way that we can handle. The transition will not be
fast and the removal of the gen_*.erl modules is certainly not
one of the most important ones as I see it.
> 
>   Then  all the  remaining modules  should  be re-written  so that  NO
> references are made to the stuff that is removed.
> 
>   Finally we  will be able  to give an  exact and final list  of every
> module in  the core with  a written argument that  motivates *exactly*
> why it in the core.
> 
>   When this has been done I will be happy.
> 
>   IMHO the  average programmer should  be able to easily  remember all
> the Erlang primitive and know about *all* the core modules do - Erlang
> should and must be  a *small* langauge - that is easy  top learn - the
> core libraries should have the same properties.
> 
>   I can  volunteer to do this (I  can't do Jocke's big  project) - but
> I'd need help (i.e.  volunteers to *remove* non core-dependencies from
> dets and gen_tcp etc.).

I don't agree that the removal of e.g gen_server and the dependencies to 
gen_server is top priority when it comes to improve and clean up in 
kernel and stdlib.
Since many years we have treated kernel and stdlib as one application 
since the interdependencies are impossible to handle. A way to
solve this step by step is to move out modules that don't belong in
kernel+stdlib one by one util we have a useful result.

> 
>   All this  pre-supposes that the OTP  people like the idea  - since I
> don't want to get into the "two systems" argument.
> 
>   For those  people who like the  "big and everything  release" we can
> bundle the  core with  *all* the applications  - if that's  what turns
> them on.
> 
>   I view  recoding and re-organizing the  core as *essential*  - in so
> much as this will yield a tighter and more easily maintained system.
> 
>   Actually the  job is easier  than you imaging  - you first  throw out
> *everything* you cannot explicitly motivate  - then run - xref - this
> tells you  which modules  you have to  re-write. The modules  you have
> removed you  move to a  set of libraries  outside the core,  and then
> re-write them using stuff *inside* core.
> 
>   Comments?

In principle ok, but in practice we have to be very careful and take 
this in small smart steps where we can keep the backward compatibility 
for all relevant interfaces as well as sequre the quality.


> 
> 
> 
> 	/Joe
> 
> 
> 
>>It would be nice if the build process asked what you wanted to build.  It
>>is a pain to be waiting for your shiny new emulator to arrive and see that
>>it's wasting your time building Corba tools.  I've adopted the procedure
>>of first touching lib/cosEvent/SKIP etc.
>>
>>	-Vance
>>
> 

/Kenneth (Product Manager of Erlang/OTP)
-- 
Kenneth Lundin              Ericsson AB
kenneth@REDACTED     ÄT2/UAB/UKH/K
			    BOX 1505
+46 8 727 57 25 	    125 25 Älvsjö