Packages in Erlang: new documentation

[Richard Carlsson]

On Thu, 4 Sep 2003, WILLIAMS Dominic wrote:

> Richard Carlsson wrote:
>
> > I absolutely agree: the . prefix notation is not recommended unless
> > you really need it. Using -import(Module) is much better. Maybe we
> > should also add a declaration -import_as(Module, Alias).
>
> But later, when I described the Eiffel renaming scheme, you wrote:
>
> > Yes, but it does not work in Erlang, because there your _cannot_
> > rename a module, since you cannot know if someone is going to
> > call it via apply(M, F, [...]) or spawn(M, F, [...]). It is not
> > possible to know in general what the M:s are, or even to which
> > M the caller is actually referring (the first M, or the later added
> > one that was renamed?)
>
> Hmm. You suggested -import_as(Module,Alias) yourself. Is that any
> different? Or did you suggest it before having thought of those
> difficulties?

Ah, but these two are very different things. When you use a declaration
"-import(package.module)" in the package system as it is today, this
only affects the expansion of explicit remote calls on the form
"module:function(...)", which are rewritten by the compiler to
"package.module:function(...)". A declaration
"-import_as(package.module,alias)" would only change the expansion
(within that particular module) to "package.alias:function(...)".

In short: this works because it only happens in the specific syntactic
context of a remote call. Other atoms are not affected, so the
following does *not* work:

	-import(my.own.foo).
	...
	   M = foo,
	...
	   M:bar(...)

because the compiler cannot (in the general case) know that the
occurrence of the atom 'foo' will be used as the name of a module
and nothing else. It is unfortunate that we have to rely on the
context in this way, but it is a consequence of how the Erlang
language was constructed, and there is no workaround. There is a
similar problem with local function calls. The following is correct:

	X = my_function(Y)

(if my_function/1 exists), but the following will cause a
runtime error:

	F = my_function,
	X = F(Y)

because the *value* F is just an atom, not a function. The first case
only works because the compiler has recognized the syntax
"<atom>(A1,...,An)" as a special case, and rewritten it to (in effect)
"(fun <atom>/<arity>)(A1,...,An)" when going to the core language.
(Note that if Erlang had used variables for function and module names,
lexically speaking, instead of atoms, we would not have these problems.)

In the suggested case of dynamic renaming in spawn/apply, however, the
problem is that there is no context at all! We have no type system that
can tell us that a particular value is a "name" that can and should be
undergo renaming. Names are just atoms - and in some cases they are even
strings that become concatenated and then turned into atoms. So if one
module passes an atom to another module (maybe through a long chain of
indirections, storing in data structures, etc.), and the recipient takes
this atom and uses it in a call to 'spawn(M,F,[...])', we cannot tell
whether the atom ought to be renamed according to the local renaming
table of the module that calls 'spawn', or according to that of the
module where the atom originated. And we can't just rename an atom
unless we can prove that it will _only_ be used as a module name in
calls - what if the atom is _also_ used for other things, such as naming
a server, or being part of a file name? This is why, in the final
expanded code, all module names need to be "absolute references", i.e.,
fully qualified.

> Anyway (Eiffel philosophy again), shouldn't things be hard for
> compiler programmers, not application programmers ? ;-)

Trust me on this: they already are. ;-(

> Oh, and I very much share the concern voiced by others about tying in
> the package structure to the filesystem.

Well, I have to say like Ulf did that I just don't know any other good
solution. The precise connection between the naming of packages and the
storage of files can always be changed by hacking the code loader. But I
think that anyone who considers subdirectories to be a bad idea for
storing code should first of all take some time to think about why
typical file systems work the way they do, and why they have not yet
been replaced with "something better".

For example, the HiPE compiler consists of some 150 modules. I would
like to *not* have to have the object files in a single directory. We
already have the source files in separate subdirectories, for the sake
of sanity. I fail to see what is so good about a single directory
stuffed with hundreds of files with very long names. I agree that search
paths are evil. The Erlang/OTP default search path contains about 45
entries! And yet all of these are subdirectories of the "lib" directory.
With packages, it could be just a single entry in the path.

	/Richard


Richard Carlsson (richardc@REDACTED)   (This space intentionally left blank.)
E-mail: Richard.Carlsson@REDACTED	WWW: http://user.it.uu.se/~richardc/
 "Having users is like optimization: the wise course is to delay it."
   -- Paul Graham