[erlang-questions] Why do we need modules at all?

[Tim Watson]

> Now I find this very convenient when I write a new small utility function
> I stick in in elib1_misc.erl - no mental anguish in choosing a module
> name is involved.

But it's not very helpful to someone else who's going to have to
mentally parse your lib_misc library to figure out what they're
looking for. It does sound a bit like a dumping ground.

>
> The observation that I find this very-convenient is telling me something
> about modules - I like my elib1_misc it feels right.
>
> (aside - It seems many development projects have their own private
> lib_miscs ...)

Perhaps if those libraries were broken into their constituent parts
and then those into their constituent projects, they would be easier
to re-use. I see the same thing on many projects - a
<project-name>_config.erl that wraps a gen_server with maybe some
calls to application:get_env and a <project_name>_log.erl and so on. I
nearly wrote a erl-config package on github until I realised gproc
would be a better choice for my needs.

>
> Which brings me to the point of my question.
>
> Do we need module's at all? Erlang programs are composed of lots of small
> functions, the only place where modules seem useful is to hide a letrec.
>

I think that's an oversimplification. Compilation units (e.g., putting
code in different files) is beneficial for humans first and foremost.
I think that categorising functions based on subject-area and/or the
process they relate too is very useful. Once you do that categorising,
putting all the categories in once place is very sensible. OCaml and
Haskell programmers (of which I'm only experiences with the former),
use letrec (and similar concepts such as Haskell's `where`) heavily,
but would not dream of doing away with our module systems just because
we can encapsulate things in this way.

Modules are a useful categorisation for the human mind to remember
things by. Like packages, they help us remember where to look for
things. They're also useful for hiding - breaking encapsulation is a
choice, not something you cannot help when designing a module.

> The downside is we have had to *invent* one module name math - whose *only*
> purpose is to hide the definition of fib/3 which we don't want to be made
> callable.
>

Actually if you put all your mathematical functions in this module, it
would be more useful (and easier to remember).

> If we put a second function into the module math, then this second function
> could call fib/3 which breaks the encapsulation of fib/3.

Not sure that's such a terrible thing really.

>
> We could say:
>
> let fib = fun(N) -> fib(N, 1, 0) end
> in
>    fib(N, A, B) when N < 2 -> A;
>    fib(N, A, B) -> fib(N-1, A+B, A).
> end.
>

Having letrec (or where) is an awesome idea. Don't actually care much
about the choice of syntax - just another one to learn.

> Does the idea of a module come from the idea that functions have to be
> stored somewhere, so we store them in a file, and we slurp the
> file (as a unit) into the system, so the file becomes a module?

Probably, yes. It is convenient and familiar to do so.

>
> If all the files were store by themselves in a database would this
> change things.
>

Not really, as long as the database is always available and I *never
ever* run into a situation where I can't work because I'm not
connected.

> I am thinking more and more that if would be nice to have *all* functions in
> a key_value database with unique names.

I love the idea of a massive code repo/server, but having everything
in one namespace sounds like a nightmare.

> The unique names bit is interesting - is this a good idea. Qualified
> names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good
> idea but but when I'm programming I have to invent the xxx or the
> a.b.c which is very difficult. It also involves the "decision problem"
> if the namespaces xxx and a.b.c already exist I have to *choose* which
> to put my new function in.

Still I think it's the right thing to do.

>
> I think there might be a case for alises here joe:foo/2 could be used
> while developing "joe" would expand to a horrible random local string the
> real name being ab123aZwerasch123123_foo/2  but I would not be able to
> publish my code or make it available to a third_part before I had
> chosen a sensible name.

Yuk. ;)

>
> (( managing namespaces seems really tricky, a lot of peoople seem
> to thing that the problem goes away by adding "." 's to the name
> but managing a namespace with namees like foo.bar.baz.z is just as complex
> as managing a namespace with names like foo_bar_baz_z or names like
> 0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic
> name like www.a.b to a reference like 123.45.23.12 - but how do we discover
> the initial name www.a.b? - there are two answers - a) we are given the name
> (ie we click on a link) - we do not know the name but we search fo it ))

You're right it is hard, but it is possible.

>
>
> When programs are small we can live with "just the code" in "a few
> modules" the ratio of code to meta data is high.
>
> When programs are large we need a lot of meta-data to understand them.
>
> I would like to see all functions with all meta-data in a data base.
>
> I'd like to say:
>
>    lookup(foo,2,Attribute) when Attribute =
>
>       code|source|documentation|type signatures|revision history|authors|...
>

I like the idea of finding a function based on a wider range of
attributes such as these, but not choosing the route of dispatch based
on a loose predicate. Lookup/3 would have to guarantee that it will
resolve to a single callee every time, which I can't see being
possible.