[erlang-questions] Updates, lenses, and why cross-module inlining would be nice

[Richard A. O'Keefe]

On 27/11/2015, at 6:30 pm, Michael Truog <mjtruog@REDACTED> wrote:
> Having it be opt-in does sound good, but it should complicate
> hot-code loading, making a module update possibly fail due to the
> import_static dependency on an external module.

We already have precisely this problem with headers.
Header dependencies are not tracked by the code loading
support.  If modules A and B rely on header C to
describe their interface, and C changes and B is updated
to match, the fact that A doesn't match any more is not
detected.

At least with an import_static dependency *stated* and
right there in the .beam file (as it should be) you can
*tell* when you have a problem.

>  If module 'a' uses
> import_static on module 'b', does the inlining (actual source code) of
> module 'a' change during runtime if module 'b' changes?  When the
> module 'b' update indirectly alters module 'a', does its module
> version not change?  If you were allowed to do an update of a
> module used for inlining, it could create a ripple-effect in the system,
> making the process more brittle and prone to failure, due to the
> lack of isolation.

Headers are brittle and prone to failure in precisely this
way right now, it's just that the system doesn't notice.

Hmmm.  The lens module could perfectly well be a header.
Instead of

get({G,_,_}, X) -> G(X).

in a module, we'd have

-define(GET(Lens, Datum), case Lens of {G,_,_} -> G(Datum) end).

and so on.  Would this change the actual number or character of
dependencies?  No.  What would be the effect of changing a
lens.hrl file instead of a lens.erl file?  Just the same breakage
except that the system wouldn't notice and would not be able to
reject a load telling you something would have broken.

It would of course be possible to provide a parse transform
for any specific module to accomplish the same effect as
cross-module inlining, but this has *precisely* the same
dependency effect as cross-module inlining or headers.

"We have already determined what you are madam, now we're
haggling over the price."

I believe that dependence on a parse transform or a header
should be tracked in .beam files and that if you try to
load a .beam file the system ought to be able to tell you
"this has a dependency which has since been changed, why
not recompile first?"
> 
> If instead the modules that are inlined are marked "permanent" and
> prevented from being updated in the future, then there is a reason
> to not use inlining

I am not with you.  How does making inlining *safe* become a
reason to *not* use it?

It's not as if I'm proposing anything that wasn't already known
in the Lisp world back in the 70s.
> 
> You may be able to make a lens implementation that relies on
> header file usage,

That's rather like saying "You could avoid the problems of
playing Russian Roulette with a pistol by using a fully
loaded shotgun instead."

> which would allow all the functions to be within a
> single module and allowed to be inlined there.  That approach
> wouldn't cause problems for module updating in the future.

It would cause *precisely* the same problems that headers
have always (at least potentially) caused.

m% cat goo.hrl
-define(GOO, 42).

m% cat foo.erl
-module(foo).
-export([foo/0]).
-include("goo.hrl").

foo() ->
    boo:boo(?GOO).

m% erlc foo.erl
m% ed goo.hrl
s/42/137/
w
q
m% cat goo.hrl
-define(GOO, 137).

m% cat boo.erl
-module(boo).
-export([boo/1]).
-include("goo.hrl").

boo(?GOO) ->
    ok.
m% erlc erlc boo.erl
m% erl

1> l(boo).
{module,boo}
2> l(foo).
{module,foo}
3> foo:foo().
** exception error: no function clause matching boo:boo(42) (boo.erl, line 5)
4> c(boo).
{ok,boo}
5> c(foo).
{ok,foo}
6> foo:foo().
ok

RIGHT NOW you can get mysterious errors that go away
when you recompile things, due to the use of headers.
import_static doesn't increase the problems, or the
amount of work you have to do to fix them, or the nature
of that work.  All it does is give the system a chance
to *notice* that the problem already exists.

ANY use of information that is not physically present in
the source file, whether it comes from using Leex or Yecc
or the preprocessor or a parse transform or anything else
is a *dependency* that needs to be tracked.