Proposed change to libraries

[Richard A. O'Keefe]

Kostis Sagonas <kostis@REDACTED> wrote:

	First let me enumerate the things we (all?) agree upon:
	
	 1. Something needs to be done in libraries to make them more consistent
	    with their documentation and allow more effective type checking.

I disagree, in the sense that it is not *JUST* the libraries that come
with Erlang/OTP.  What I would like is something *easy* that people can
do to their *own* code to make it better fit the intended types.

The important difference here is that one's view of the scope of the
problem affects one's judgement about how much extra effort it is
tolerable to impose.

If you think it is _just_ the libraries, and the libraries are "owned"
by a small disciplined group of people, then even something fairly
heavyweight could be tolerated and we could have confidence that the
job would be completed.

If, with me, you think that we *all* write code that would benefit from
better checking, you want a solution that is very very easy to apply.

Also, we need a new edition of the Erlang book and the style guides which
explain the new recommended style.  If the new style is not to drive people
away in fits of contemptuous laughter, it again has to be a solution that
is very very easy to apply.  (Like Common Lisp adding ENDP so that you can
"fix" a list iteration simply by changing (NULL x) to (ENDP x).)

	 2. Tuples-as-funs need to be handled somehow but only for a
	    transitional period (of currently unspecified length ;-).

If we agree on that, then there should be NOTHING in the code which
explicitly tests for {M,F} tuples.

	 3. The checks that will be performed for tuples-as-funs cannot be as
	    strong as (at least some of us) would like them to be.
	 4. Some new built-in guards have to be introduced.  If they handle
	    tuples-as-funs, this handling will be taken out when their support
	    is discontinued.
	 5. It is probably not a good idea to modify the semantics of existing
	    guards (e.g. is_function/1) to also handle tuples-as-funs only to
	    take these modifications away some fine day.

Agreed on those.

	So, here is an alternative proposal that satisfies the above
	constraints and allows us to do more effective type checking.
	Modify map/2 to look as follows:
	
	    map({M,F}, L) -> map(fun(X) -> M:F(X) end, L);
	    map(F, [H|T]) -> [F(H) | map(F, T)];
	    map(F, []) when is_function(F,1) -> [].
	
Presumably this is a mistake for

	map({M,F}, L) when atom(M), atom(F) ->
	    map1(fun(X) -> M:F(X) end, L);
	map(F, L) ->
	    map1(F, L).

	map1(F, [H|T]) -> [F(H) | map1(F, T)];
	map1(F, []) when is_function(F, 1) -> [].

But that does NOT satisfy the above constraints.  Not only does it not
satisfy constraint 2, it jumps up and down on it and then spits on the
bleeding corpse.  It does exactly the same thing to my modified
constraint 1:  the solution must be very easy to apply.

A solution which requires splitting existing functions into two and
adding at least two more clauses is a heavyweight solution.  The OTP
team may be disciplined enough to do this, but most other people will
say "I've been managing without type checking up to now, this is a
huge amount of boring and error prone work that gives _me_ very little
immediate benefit."

The proposed change to library functions is
(1) heavyweight.  One function turns into two.  Two new clauses are added.
    (More if there's more than one function argument.)
(2) error prone.  It's far too easy to write map (or map1) instead of
    map1 (or map).  It's FAR too easy to forget to change _all_ the
    existing recursive calls to map.
(3) a huge roadblock in the way of removing support for {M,F} pairs;
    every single function that has been modified in this style is another
    member of the resistance fighting against such a change.  Remove it
    from the libraries, and it will still be there in some user code.
(4) a *double* cost.  All such functions have two be modified on two
    occasions.  Now, in order to add the is_function/2 guard without
    removing support for {M,F} pairs.  And later, to remove support for
    {M,F} pairs.  Every single higher order function.  Change to every
    line of every such function.  On two separate occasions.

Now if we adopt is_applicable/[1,2], we find
(1) the change is lightweight; a guard is _added_ to base case clauses
    of higher-order functions but it's only a matter of inserting text,
    not altering any single text.  No functions are split, no clauses added.
(2) Not particularly error prone.  The most likely error is forgetting to
    make the change.
(3) A positive help when support for {M,F} pairs is removed.  Just change
    is_applicable/[1,2] to be the same as is_function/[1,2] and that *one*
    alteration makes the change *everywhere*.
(4) A *single* cost.  The code is changed once.  If it is decided to
    replace is_applicable/[1,2] by is_function/[1,2], that can be
    automated, but there really isn't any need to do that.

	and then is_function/2 only handles proper fun objects.  Besides being
	much cleaner, I think, it also has the extra advantage that calls to
	map/2 with a tuple-as-fun also work when compiled in native code (which
	currently they do not).

Ah.  So this is really all about patching the source code to work around
a bug in the compiler!

	Then taking out support for tuples-as-funs is
	just a matter of removing the extra clause -- the semantics of all guards,
	current or new, remains unchanged.
	
Replacing one guard (is_applicable/2) by another (is_function/2) is
easy to automate reliably.  Removing the extra clauses is _not_ so
easy to automate reliably.