Child modules draft feedback wanted

[Romain Lenglet]

Richard A. O'Keefe wrote:
> Romain Lenglet <rlenglet@REDACTED> wrote
> a reply to my request for comments.  Unfortunately, he
> sent two copies, and I replied privately to what looked like
> private mail before seeing it again in this mailing list.
>
> I enclose a copy of my reply.  In brief, his counter-proposal
> doesn't seem to do anything for most of the problems I am
> trying to solve.  PERHAPS I AM TRYING TO SOLVE THE WRONG
> PROBLEMS. My basic question is "how can I get rid of -include
> and -ifdef and system-dependent file names in source code?" 
> Maybe that's the wrong question.

No, I am trying to solve that problem: removing the use of 
-include, .hrl files, and -ifdef when those mechanisms are used 
to select different alternatives in an architecture / 
configuration.

But my hypothesis (which I should have stated explicitly, sorry) 
is that architecture alternative selection should be done at the 
granularity of modules (not arbitrary pieces of code). This is 
more restrictive that the -include/-ifdef mechanisms, since you 
must encapsulate every alternative in modules/functions, but 
such a requirement has never been a problem in existing 
component models. It even has a good effect on code structuring, 
and eases maintenance.

I am not trying to adapt the include mechanisms as you do (by 
introducing the concepts of "parent" and "child" modules), but 
rather to remove that concept entirely, and use only the 
existing concept of modules, and a way to specify dependencies 
between modules in a flexible way, so as to allow to easily 
select architectures without modifying the code.

> Your proposal is to add just one construction,
>
>     -require(Logical_Module_Name, Imports).
>
> Except for this being a "logical" module name, it is not clear
> whether or how this differs from the existing -import
> directive.

Yes, that is the only difference with the -import module. The 
advantage over import is that the module does not depend on a 
particular real module statically. That binding can be stated 
outside of the module. With -import, the binding is static.

>       My -require clause is almost identical to your -use_child
>       directive, and the module name is not a real module name, 
but
>       rather a "logical" one which scope is limited to the
> declaring module.
>
> But this now GREATLY complicates the implementation.

It can very easily be implemented by program transformation at 
compile time. No need to change the runtime.

> In my proposal, I was extremely careful to ensure that NOTHING
> about the way module: prefixes currently work would have to
> change. There would continue to be a single flat global name 
> space for modules and the interpretation of a module name
> would NOT be in any way context- dependent.  [Added in this
> copy:]  I want it to be as easy as possible to add the new
> constructs to Erlang; the more that changes can be limited to
> the compiler, with *no* change to the semantics of full
> modules as such, the happier I will be.

Replacing a "logical" module name in module:call() statements in 
the scope in a module, by an actual module name, can be done at 
compile time very easily.

> You are offering something as a feature that I regarded as a
> serious problem to be avoided.

I don't see where there is a serious problem.

>       And an ADL (yes, the "configuration language" here is in 
fact
> an Architecture Description Language) would allow to specify
> the dependencies *outside* of the code, as bindings between
> the -requires clauses' logical module names and actual module
> names.
>
> Either you have missed the point of my configuration language
> or I have missed the point of your ADL.  I am trying to solve
> two problems with the (as yet unwritten-up) configuration
> language:
>
>     (1) I am trying to remove *FILE* names from the source
> files. I don't see where you say anything about file names.

Since module names are also file names, dealing with logical 
module names instead of real module names removes file names 
from the source files.

>     (2) I am trying to provide an alternative to -ifdef.
>         Your ADL (no more described than mine) would appear to
> deal with that issue.

As I understand it, the main use of -ifdef, etc. is to select 
alternative architectures. Using an ADL, you would simply have 
to write several specs.

> There's an anti-point:
>
>     (3) I *don't* want dependencies stated outside the source
> code; I want dependencies very explicit *in* the source code.

*Dependencies* (what is required) must be stated in a module's 
source code, since they are very interdependent on the module's 
implementation (you cannot change one without changing the 
other).
On the other hand, *bindings* should be stated outside, because 
the architecture (the configuration) is a concern separated from 
the modules implementation.

[...]
> Maybe we are using the word "dependencies" to mean different
> things.

Maybe.

>
>       And 'actualfred' would be a normal module.
>
> No, that's precisely what I *don't* want.  I am NOT trying to
> set up some kind of hierarchy of normal modules.

Modules and their bindings form a "flat" graph: they do not 
necessarily form a tree, or "hierarchy".

> I am trying 
> to design a principled replacement for -include AND I am
> trying to meet a frequently expressed "need" for some kind of
> hierarchical name scope WITHIN a single module.

And this contradicts my basic principle (which I state in the 
beginning of this email, sorry I should have stated this 
before): configuration alternatives ("what can be required") 
should be specified as functions in modules. We already have 
functions, and modules, and they can be used as units for 
selecting configuration alternatives. We don't need to introduce 
new concepts ("parent" and "child" modules) only for that 
purpose.

> In particular, there are *TWO*-way links between an -include
> file and its host, and that's why -use_child and -begin_child
> have *two* interface lists.

No. There *may* be two links. Or more (to other modules, etc.).
And those links can be expressed in a unique simple way, with my 
-require clause. Our proposals are equivalent, except that 
-require is more general.

> You seem to be taking the present system of modules and adding
> two things: (1) a module might be known by different names in
> other modules (2) the mapping "In real module X, logical
> module name Y means real module Z" is expressed outside the
> source files.

Exactly.

> It is as if you are asking "How could we make the present
> system more flexible?"  But I am saying "the present system of
> relying on -include files is intolerable; how can we replace
> -include?"

My system replaces -ifdef and -include when it is used to select 
configuration alternatives. It replaces that (taken from your 
own document):
-ifdef(use_ping).
-include("ping.hrl").
-else.
-include("pong.hrl").
-endif.
When ping.hrl and pong.hrl contain alternative implementations.

>       I believe that we don't need any more concepts in the ADL 
so
> far. Except perhaps your concepts of replaceable / integrated.
> But I believe that such things should be specified in an ADL
> spec rather than in the code:
>
> That won't work.  Remember, the heart and soul of my proposal
> is GETTING RID OF -include.  There are things that -include
> can do which normal imports CANNOT DO AT ALL.  Many of them
> are things that shouldn't be done.

I agree very much with you.
And my proposal does not replace all uses of -include / -ifdef, 
but it solves at least the important problem of selecting 
alternative implementations.

> Long term, we might even 
> hope to replace -record with psi-terms or abstract patterns. 
> But short term we are stuck with -record.

I also agree with you, in that my proposal does not replace 
-record.

What I would like is to introduce one construct for every actual 
usage of -include / -ifdef, until all uses are covered by better 
constructs, and we can get rid of / deprecate -include and 
-ifdef. I think that my -require proposal covers well the 
problem of selecting alternative implementations. We still need 
to find a replacement for records, etc.

I think that it is a bad idea to try to specify a single 
construct that would replace all actual usages of -include and 
-ifdef: either the new construct will be as bad as what it 
replaces, or it will not cover all usages.

Maybe you have arguments against that, but please write them. ;-)

> And we need to bind 
> a module to *some* kind of entity which can declare records,
> so that the module can use those record declarations.  Your
> -require only provides a way to import functions from a
> logical module.  That means that the -record problem is left
> RIGHT WHERE WE STARTED and -include is STILL needed to solve
> it.

I agree. But let's solve one problem at a time?!

> This creates an absolute distinction between integrated child
> modules and full modules:  integrated child modules *CAN*
> provide -record information to their host, but full modules
> CANNOT.

Why not have a specific construct or convention to get record 
information at a module level (not at a source code level, like 
-record). For instance, for behaviour, the functions to 
implement are specified in the behaviour module as a function: 
behaviour_info/1. What not have a similar record_info/1 
function??? That function would return the list of field names 
(as atoms) for the record name given as an atom as an argument.

E.g.:

-module(record_spec).
-export([record_info/1]).
record_info(record1) ->
    [field1, field2];
record_info(record2) ->
    [field1, field2, field3].

And in using modules, we could use a modified -record construct:
-record(Module, RecordName).
It would be like an "import of record definition". E.g.:
-record(record_spec, record1).

We all are familiar with such constructs, support for such 
constructs already exists in the compiler (for behaviours), and 
we get rid of the need for -include for records.

What do you think of that? A specific solution for a specific 
problem: records specifications. And my -require proposal covers 
the usage of -include, etc. for alternative selection.
Are there other usages that must be covered?

> The fact that you *need* a record definition from 
> somewhere is something that should be stated explicitly in the
> source code.  The distinction between -record declarations and
> ordinary functions drives a distinction between child modules
> and full modules.

Doesn't my proposal just above meet that requirement?

>       IMHO, my proposal:
>       1- is simpler than your proposal: only one new clause is
> added to the syntax (-require);
>
> Yes, but it's simpler because it solves at most one of the
> problems I am trying to solve.

That's right.

>       2- does not distinguish between "parent" and "child" 
modules:
>       such a distinction is not necessary;
>
> But as I have just argued, it IS necessary.

Again, I don't think so.

>       3- allows all that your proposal allows.
>
> It doesn't seem to allow hardly anything of what my proposal
> allows.

But you seem to try to cover all usages of the preprocessor with 
a unique construct, which I think is a bad idea, and forces you 
to introduce new concepts such as "parent" and "child" modules.
I propose instead to replace every usage by a specific solution.

> On the other hand, my proposal doesn't allow what your
> proposal does. My proposal does not allow a full module to be
> known by different names. I'm not sure how badly I want to
> prevent that, but it's certainly not a solution to any problem
> I'm trying to solve.
>
> That sounds rather negative.
>
> + Thank you VERY much for thinking about this.
>   Even if we don't agree, the next version of the draft will
> be improved by me trying to clear things up.
>
> + My proposal certainly could be simplified.  (In particular,
> there is no actual necessity for in line children, although
> other languages have them and they do meet a frequently
> expressed desired for something like block structure.)  It is
> entirely credible that something not wholly dissimilar to your
> proposal COULD do the job.

-- 
Romain LENGLET