[erlang-questions] Are packaged modules the way to go ?

[cyril Romain]

Hi,

Bengt Kleberg wrote:
> I belive it shows that the existing packaged modules are not a
> sufficiently good idea to warrant official adaptation.
>   

Indeed.... Thanks for the link, it's a long but worth reading thread!


After a deeper look, here is my feedback about packaged modules. I'll 
give objective facts first, before somewhat subjective notes about 
packaged modules.


_WITHOUT PACKAGED MODULES_

A project without packaged modules:
 - has a flat module structure (i.e. there are no modules within modules).
 - generally has, if not always, its sources files (*.erl) in one 
directory with no subdirectory. Same things for binary files (*.beam).
 - the name of a module is the name of the Erlang file (without 
extension) and vice versa.

And consequently:
 - tends to use the following name convention to avoid file/module name 
clashes: myapplication_mycomponent_mymodule.erl (sometimes there is no 
mycomponent, and more rarely no myapplication in the name). 
Abbreviations such as myapp instead of myapplication are often use to 
avoid very long name.
 - can use macros to avoid using long module name within the code. E.g. 
-define(MYMOD, myapplication_mycomponent_mymodule), and calling 
?MYMOD:foo().
 - the project file structure makes Emakefile quite straightforward to 
write. A one liner Emakefile may even suffice: {'src/*', [debug_info, 
{outdir, "ebin"}, {i, "include"}]}.
 - tools analyzing Erlang files or project file structure [1] can rely 
on the following statement: the file name is the module name and vice versa.


_WITH PACKAGED MODULES_

A project with packaged modules:
 - has a tree module structure.
 - generally suggest (resp. require) the source (resp. the binary) file 
structure to follow the tree module structure.
 - the name of a module can differ from the file name (the full module 
name is actually a part of the file path, with . instead of /).
 - a module in a package requires additional import of the top-most 
modules used in the code. Indeed you need to either:
    - use the prefix . to specify that you are referring to the root 
package.
    - add an 'import' statement at the top of the module.
E.g. to use the standard library 'lists' module: you would either use 
'.lists:reverse' instead of 'lists:reverse', or you have to -import(lists).
Anther example if a module in a package defines -behaviour(supervisor), 
you have to -import(supervisor) or to use the . prefix to call 
supervisor functions.

And consequently:
 - short file names can be used. mymodule.erl can actually defines 
-module(myapplication.mycomponent.mymodule).
 - make code more readable and easier to maintain, because no need to 
specify the namespace when calling function defined in mymodule.
 - require more complex Emakefile to dispatch .beam files in a directory 
structure following the module namespace structure.
 - allow multiple Erlang files to have the same name, as long as they 
define modules in different namespace. E.g the ~/ebin/core/net.beam, 
~/ebin/plugin/net.beam and Erlang kernel's net.beam files can coexist 
with the respective core.net, plugin.net and net module names.
 - the full module name should be read from the Erlang file to be known. 
Indeed in /a/path/to/mymodule.erl, the module name could be 'mymodule' 
or 'to.mymodule' or etc.


_NOTES_:

Packages modules suggest to the developers to split their software into 
sub-component, which is good IMHO. Project containing ton of source 
files are indeed easier to work with when split into orthogonal 
components, with one directory per component and where code dependencies 
are more clear.
Of course it can be achieved without packaged module and one can badly 
design and split their packaged modules, but packages modules just 
strongly suggest this good software development practice.

Packages modules, as they are now, have some drawbacks though:

In regards to developer's code:
     - using the prefix . for top-most module breaks code consistency 
between top-most and non-to-most modules, unless you or Erlang would 
enforce that prefix to be used everywhere.
     - additional import tends to conflict with the 'don't use import' 
Erlang programming rule [1].
However it nicely solves the too_much_long_names in code issue without 
using UPPERCASEMACROS.

Some unexpected "bugs" can also show up when using module packages. For 
instance:
     - code:where_is_code(Filename) looks for the file in code path but 
not in sub-directories, so it is unable to find packaged module.
     - code:get_object_code(Mod) can succeed while Mod:module_info() 
fails, when e.g. Mod = test.plugin.net but the module name actually is 
plugin.net.


It is also worth noting that packaged modules:
 1. does not solves name clash issues. It only substitutes name clash 
issues by module namespace clash issues. Indeed if you use for instance 
two Erlang projects using packages that pick the same name and namespace 
for a module, the code will clash.
 2. implies more complexity in tools analyzing Erlang files or project 
file structure such as debugger, dialyzer, Emakefile, etc. Indeed those 
tools cannot assume the filename is the module name and vice versa anymore.
 3. are not supported yet by some useful tools (dialyzer, Erlang 
automake, etc.). E.g. dialyzer assumes the file name is the module name 
and vice versa, and therefore fails to analyze projects with duplicate 
Erlang files name, even if those files define modules in different 
namespace.
 4. last but not least, it breaks referential transparency in some way [2].

Further discussion about packages can be found on the Erlang mailing 
list. See [3] and [4].


Although packaged modules seems a good idea, I'm still wondering if they 
are worth it due the above notes, and in regards to the Erlang code 
consistency in general.
I'm tempted to say that packaged modules are good and can enforce 
consistency, but only if widely adopted by Erlang developers and widely 
supported by tools.
Unfortunately for now it just look like it adds more confusion than 
consistency :-(

If there is a consensus about no adoption, are packaged modules planned 
to be removed from Erlang/OTP releases ?
Any official decision or thought on this subject ?

Sorry for the long mail. Hope it will be useful.
Best regards,

  Cyril

PS: for some reason Joe Armstrong does not mention packaged modules in 
his last book. Hope I'm not reviving an old flame ware here!
If you notice something wrong or that I didn't mentioned, please let 
everyone know.

[1] http://www.erlang.se/doc/programming_rules.shtml#HDR26
[2] 
http://www.erlang.org/pipermail/erlang-questions/2006-October/023749.html
[3] http://www.erlang.org/pipermail/erlang-questions/2003-April/008571.html
[4] 
http://www.erlang.org/pipermail/erlang-questions/2006-November/023768.html