[erlang-questions] Embedded vs Interactive - Why embedded?

[Ryan]

On 03/10/2016 02:43 PM, Michael Truog wrote:
> The embedded/interactive functionality really is focused on module
> loading either at startup or lazily, as you have described.  I only
> mentioned initialization and configuration source code, due to how this
> fail-fast concept can be applied to source code.  While it may seem that
> the embedded/interactive choice is not an important one, with execution
> generally happening in the same way, it can be important due to some
> code paths being infrequent and problems with the dependencies like
> modules with the same name (and unfortunately sometimes it then depends
> on the search directory order, which can lead to problems during
> execution that are counter-intuitive).
>
I can totally understand the "modules with same name" problem, being 
from a Java background, where the same problem happens with class names. 
In this thread, a fellow named Max Lapshin indicated that he's using 
interactive mode and loading all modules at startup. Do you think that 
doing that would meet this concern of yours, assuming that during 
loading, one could detect if there were multiple modules of the same 
name on the code path? I'm not sure how to do that, but surely there's a 
way.

> I meant using interactive mode for manual usage of the Erlang shell, not
> real testing of a release.  Only using interactive mode for development
> testing of random segments of Erlang source code.  Even that usage of
> interactive mode can be problematic due to the undocumented differences
> between the Erlang shell execution and normal Erlang module execution.
> So, all releases for testing and production should be real releases
> running in embedded mode.  The interactive mode just helps you quickly
> use the Erlang shell to check stuff.
>
I think this gets to the heart of what I'm trying to figure out: are 
there "undocumented differences" between interactive and embedded mode 
that are going to affect a long-running, production deployment? I'm 
assuming that what you refer to as "normal Erlang module execution" is 
embedded mode, and that's what this thread is all about. Why is embedded 
mode considered "normal"? Why can't we just run interactive mode 
everywhere? Assuming your concerns about dependencies can be addressed 
in a fail-fast way, is there really a good reason not to?

> That can be weird.  I know there can be problems with reltool including
> dependencies that are not dependencies of the main application, due to
> xref being used internally by reltool instead of just looking at the
> .app dependencies.  That only affects using applications dynamically
> though, and doing that is uncommon. Normally all the Erlang applications
> are part of a static hierarchy and you only use a single boot file
> during the lifetime of the Erlang VM.
>
Ahh, xref would explain it. I think you're kind of proving my point 
about dependencies, though. If I can concretely say that my dependency 
list is [X,Y,Z], then why introduce the complexity overhead of the 
erlang release tools? They only seem to obfuscate things and introduce 
more opportunity for mistakes. They certainly have some surprising 
behavior built in. If I know my dependencies, then I should put the same 
set of dependencies in all environments, and load the code the same way 
everywhere. Isn't that the most foolproof way to ensure no surprises?

As to the boot file, I haven't tested extensively, but I believe you can 
use a boot file with either embedded or interactive mode, and it works 
the same way except for the eager/lazy module loading.

> Yes, release building is a build-time concern, but making sure the
> release is ran in a dependable way is what relates to the
> embedded/interactive mode decision.  Always using the embedded mode when
> a release is ran will help make sure the release is executed
> dependably.  You may have the initial startup cost of loading all your
> modules due to embedded mode but that delay is very small with the
> Erlang VM and I have never seen it as a problem (even with an ARM and
> slow SSD memory).
>
I'm not concerned with the initial loading cost of modules. I'm 
concerned with bugs going to production. I think what you said here is 
what's the primary thrust of my question: "Always using the embedded 
mode when a release is ran will help make sure the release is executed 
dependably."

That's the sort of assertion I've found scattered around the net, and 
that's what I'm questioning here. How do you know that? How can embedded 
be more dependable than interactive when the only difference is in eager 
vs. lazy module loading? Keep in mind that we agree that dependency 
management is a *build-time* concern, so the proper dependencies are 
theoretically guaranteed to be in place by the time the system is 
started. How is the runtime mode of the system going to affect its 
dependability?

For follow-up, can you give me some concrete example(s) in which running 
in embedded mode would (did) prevent some issue that running in 
interactive wouldn't catch?