[erlang-questions] Package Support/Use

[Dominic Williams]

Hi Eric,

> Shouldn't that debate happen then? Having this quasi
> experimental aspect of the language that some find useful
> and some don't that may or may not go away seems to be a
> problem.

I agree, although in fact the debate keeps coming back. I'll
reiterate the points I made several years ago, and then I
suppose I should put my energy into providing the
alternative solution ;-)

Sean pointed out:

> It looks a bit like Java and is therefore not to the taste
> of many hardcore Erlang hackers.

Yes, but it's not just snobbery: I worked for years with C++
namespaces and Java packages, and they have problems that I
would not like to relive with Erlang.

The current Erlang package system confuses several issues:

 - avoiding module name clashes

 - grouping related modules together logically
   (semantically)

 - grouping related modules together physically (file
   system)

NAME CLASHES
------------

Using packages to avoid name clashes has two flaws:

1) it only reduces the likelyhood of a name clash, and if
   one occurs nevertheless you are back to square one;

2) it forces you to deal all the time with a problem that
   may never occur, or rarely occurs.

Both of these are because the problem is being dealt with at
the wrong time. Name clashes should not be avoided before
they occur, they should be resolved if and when they occur.

SEMANTIC STRUCTURE
------------------

There is something to be said for being able to use the same
name in different contexts, being able to distinguish them
when necessary but not bother when in a given context. For
example, Graphic.Train (a graphical UI object) and
Radio.Train (representing a radio connection with a train):
it's nice to call them both Trains, and not to need to
specify Radio all the time in the rest of the Radio code.

This comes up a lot in object-oriented designs, but in
Erlang, I am not so sure. Modules don't map to
classes. Modules are already closer to what packages can be
used for from a design perspective.

In any case, the name clash avoidance situation comes and
spoils this otherwise legitimate use of packages, because:

1) to further reduce the chance of clash, no one resists the
   temptation of introducing additional levels (company
   name, product, ...) which have NO value in terms of
   describing the design and NEGATIVE value in terms of
   making the code clearer.

2) because packages encourage you to think about this kind
   of structure too soon, the package organisation is
   usually too complex and often does not even map to the
   code dependencies.

PHYSICAL STRUCTURE
------------------

Forcing the package structure to be mapped to a directory
structure is a real pain.

First, the directory structure becomes so much more
complex. It's not only in the editor (I am not an Emacs
newbie, and I use tags - although I haven't checked if they
work with packages yet), it's also all the rest: things you
do in the shell, scripts, makefiles, version management...

Richard O'Keefe argued that it's even a mistake to force any
mapping onto the filesystem (even module names):

http://www.erlang.org/ml-archive/erlang-questions/200309/msg00044.html

Ideally, I would like Erlang to have no files at all, like
Smalltalk. In a sense, any kind of mapping seems contrary to
this ideal. On the other hand, I suppose writing tools that
hide the filesystem is easier if the tools can make strong
assumptions about the organisation of files.

ALTERNATIVE SOLUTIONS
---------------------

The simplest way to resolve name clashes after they occur is
to rename one of the modules. Better support for automated
refactoring would make this easy, and I guess that covers
99% of the name clash situations.

Beyond that, the most elegant solutions to these issues were
pinpointed by Bertrand Meyer and are implemented in the
Eiffel language.

Regarding logical organisation, he points out that a
language's basic organisational unit (the class in Eiffel,
the module in Erlang) is itself sufficient to express
multiple, nested levels. To group several modules together
logically, use a module: it can provide a simplified, or
unified, or modified, or concatenated, or whatever, facade
for the group of related modules. I admit that this argument
is a bit stronger in Eiffel, where you can use inheritance
(including multiple inheritance without the pain). In
Erlang, you'd have to hand-code all the delegation of
function calls to the other modules. In practice though, you
rarely need to do this for all functions, and providing a
simplified front-end is enough.

Regarding name clashes, Eiffel has a system to resolve name
clashes once they occur: you can locally, in a class or a
group of classes, rename another class and use it under a
different name.

In Erlang, that would amount to being able to add a:

-rename("external/foo", bar).

attribute to a foo module to allow it to refer to another
foo module as bar, or as a compile option to allow an entire
application to use bar to refer to a foo module from another
application.

Note that this mechanism can do more than just solve name
clashes: it can be used to make code clearer when a module
name that made sense in its original context gets used in a
different context.

It has been argued that this mechanism would be difficult to
implement in Erlang:

http://www.erlang.org/ml-archive/erlang-questions/200309/msg00045.html

I find it hard to believe it would be impossible, though it
might require modifications in the runtime system.

Even if a renaming mechanism is too difficult to implement,
I don't think packages system in its current form is
satisfactory and I vote for it not to be officially adopted.

Regards,

Dominic Williams
http://www.dominicwilliams.net

----