[erlang-questions] Updates, lenses, and why cross-module inlining would be nice

[Michael Truog]

On 12/01/2015 08:06 PM, Richard A. O'Keefe wrote:
> On 1/12/2015, at 8:42 am, Michael Truog <mjtruog@REDACTED> wrote:
>> The header approach is preferable to make the dependency problems
>> a compile-time concern, rather than creating a runtime failure when
>> attempting a module update.  It is important to not crash running source
>> code due to errors.
> It *would* be "preferable to make the dependency problems
> a compile-time concern".  But using headers DOES NOT DO THAT.
The lens.hrl created from your lens.erl file does due to containing only
functions, that get dropped into the module that includes the header file,
making them look like local functions within the module.
The nowarn_unused_function compile directive was used to make sure
unused functions don't generate warnings, so it acts like the opposite
of an export statement.

I know it may be a little dirty, but the Erlang compiler allows it and it makes
inlining work.  You could consider it templating in Erlang in the absence of
explicitly supporting templates.  I have found this approach useful
elsewhere.
> Nor does the import_static approach "crash running source code
> due to errors."  import_static has two purposes:
> (1) to permit cross-module inlining and type inference safely;
> (2) to detect a clashing update *before* changing the state of
> the system in order to *avoid* crashing running source code.
The two (high-level) approaches I see for implementing import_static is
where either:
1) modules are made permanent to make sure the modules do not change,
so functions may be inlined between modules safely
2) modules are updated in groups and the groups are created by the
inlining between modules

If #1 is the approach, then it should be important to have an export_static
statement to explicitly allow module functions to be inlined, to avoid other
developers locking your module from future changes due to their
import_static usage.  Using the inline compile directive for this purpose
would be overloading its meaning, since it can be necessary to inline an
exported function for local use within a module.

If #2 is the approach, then the atomic update unit is probably best defined
as an Erlang application.  Then inlining between modules works, but only
between modules within the same Erlang application.  That group of modules
has a version associated with it, and can be tracked.  This is a low-level
module update, which would likely need to be coordinated like Erlang
application updates are coordinated now, just making the process
more complex than it currently is
(e.g., http://www.erlang.org/doc/man/appup.html).  The export_static
statement may be useful here, but it would depend on the details.
>
> It is precisely the header approach which *creates* the potential
> for runtime problems due to version skew issues NOT BEING NOTICED.
if you are relying on local functions changing instead of macros, the
beam code is changing in a very noticeable way, the same as functions
being modified.
>
> For what it's worth, I have a version of lens.erl as lens.hrl.
> It's not a pretty sight.
> There's heavy use of a ?LET macro to try to avoid variable
> capture, multiple evaluation, and so on.
> It relies a lot on the compiler inlining
>   (fun (X, ...) Body end)(E, ...)
> into
>   (X' = E, ..., Body, kill X'...)
> which, sadly, it does not appear to do.
>
>> The point of this, is to show the same source code can be used, with
>> inlining, and that the potential for breakage can be handled with testing
>> all the functions.  A lens interface should not change a whole lot, so it
>> can be a dependable interface that is trusted.
> Nobody is suggesting that import_static should be COMMON or
> used with frequently changed modules, only that something a
> bit more disciplined and rather safer than a horrible mass of
> -defines woukd be nice.
I understand.  My main concern is avoiding module inlining forcing
modules to be permanent (unable to be updated during runtime)
since that problem could easily grow in an Erlang system to negate
the hot-code loading benefit Erlang normally provides, making its
use harder to justify.  You can find an actor model implementation
in many programming languages, but Erlang's fault-tolerance
features are what sets it apart, so it is important to preserve those.

(I understand people like to argue about whether an actor model
is the same as what was defined in the 70s and that Erlang does
not fit the definition due to its differences.  I only mention it due to
the actor model being similar to Erlang processes.)
>
>>> RIGHT NOW you can get mysterious errors that go away
>>> when you recompile things, due to the use of headers.
>>> import_static doesn't increase the problems, or the
>>> amount of work you have to do to fix them, or the nature
>>> of that work.  All it does is give the system a chance
>>> to *notice* that the problem already exists.
>> Yes.  I believe this doesn't become a concern when tests are provided.
> If the modules in question are tested *separately*,
> the tests don't help.  If you test the modules *together*,
> you might as well *reload* them together.
Then that seems to favor the approach #2 mentioned above.