[erlang-questions] Embedded vs Interactive - Why embedded?

[Ryan]

On 03/09/2016 09:11 PM, Michael Truog wrote:
> My understanding is that embedded is preferred for any production use, 
> to make sure the system will fail-fast upon startup if there are any 
> problems loading dependencies.  The best time to have something fail 
> is when it first starts, due to its lifetime being undefined and 
> possibly infinite (ignoring heat death in the universe, and other 
> natural disasters that should be written out of legal agreements :-).  
> Ideally this concept is extended into the runtime of the server, into 
> initialization and configuration source code, to make sure the server 
> can fail-fast upon startup when the server is misconfigured, rather 
> than waiting an arbitrary number of hours or days to find out that a 
> problem exists.  This approach helps to avoid a reliance on a 
> fire-fighting mentality that becomes dependant on monitoring for 
> feedback on a system's health due to the source code being a 
> potentially unknowable black-box for some organizational reason.
>
This is a really good point, so let's clarify this. As I understand it, 
the difference between the two modes is that in embedded mode, all of 
the modules declared in your boot script are loaded at startup. To your 
point, if any modules happened to be missing, then startup would fail at 
this point. That's good. We like early failure. As far as I can tell, 
though, that's the only difference between embedded and interactive. 
After that initial "load all the modules", the next step would be to 
start all of your required applications. Whether by command line flag or 
boot script, that's going to progress the same way in either mode, 
assuming all the same modules are present. You talk about a 
initialization and configuration failing. Won't that cause identical 
issues in either mode, regardless of when the modules are loaded? I 
think all we're talking about here is whether code is loaded eagerly or 
lazily, and starting applications or processes works the same in either 
case.

> The interactive mode helps when testing, since things are loaded 
> automatically and you can have less concern about the dependencies, 
> since you are actively creating or modifying the dependencies.  In 
> production, you want an iron fist's control on all the dependencies, 
> to make sure everything is repeatable and the service is stable, 
> otherwise the development is pursuing stability in a serious way.  So, 
> that means that production usage should demand that all the 
> dependencies come from a specific place at a specific version that is 
> known and tracked, to be easily replicated, without any ambiguity.
>
Again, I completely agree that you want everything set in stone for 
production. I have a two-fold reply to your points.

First, IMHO running two different ways in dev/test vs production only 
*increases* the chance that errors will slip in. Part of my quest here 
is to make it so that dev/test environments behave as similarly to 
production as possible so as to eliminate issues before they make it 
that far.

Second, as to having complete control over dependencies in production, I 
don't think I'm talking about dependency management, just about code 
loading. What dependencies get deployed where is part of the release 
process. That's different from the embedded/interactive discussion, 
isn't it? In fact, I know that running embedded doesn't protect you from 
dependency problems because just a few months ago, we had a production 
release go bad because of a dependency issue. Again, we currently run 
*embedded mode* in production. The problem was that some application 
wasn't explicitly declared as a dependency in the right .app file, and 
it wasn't included in the .rel file, so it didn't get bundled into the 
release. The odd thing was that a few modules from that application 
*did* make it into the release. I'm not sure whether it was rebar or OTP 
that was responsible for that, but it made for a very confusing 
situation, even for a couple of fairly experienced devs who have been on 
this project for 2 years now. This same version ran *perfectly* in dev 
and test. It was only in production that the bug manifested.

Now, let me give a strong caveat that it's possible that we're 
misunderstanding something about how our release gets built, since it 
was someone else who wrote that part of the code, and he's no longer 
around. My point, though, is that dependency management is a build-time 
problem, isn't it? When you build a project, whether for testing or for 
production, that's when the right dependencies should be put in place. 
If you wait until runtime, you're too late. Stuff will crash.

Thanks very much for your reply. I really want to understand the basic 
issues here. Please let me know if I'm totally off base here. I have 
many years of dev experience, but not much with Erlang, so I'm still 
trying to find my way around.

Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160310/1cb6511f/attachment.htm>