[erlang-questions] The quest for the perfect programming language for massive concurrency.

[Fred Hebert]

On 01/31, kraythe . wrote:
> Well I think after seeing the arguments and the response of the community I
> am trending seriously towards Erlang. I will probably make the mental
> investment to learn and become good with emacs. And then move on from
> there. I may still have a ton of questions. I would still, for example,
> love to know who to 'reload' my application once it is started.
> 
> 

First of all, Erlang has two ways to call functions: local calls, and
fully qualified calls. Basically, the local calls happen whenever you
call a function internally within a module (`f(x)'), and fully qualified
calls hapen when you call a function and prefix it with its module name
(`my_mod:f(x)'). There are special cases for this around closures and
anonymous functions, but it's not necessary to understand it just yet.

For all basic code management, Erlang has what is more or less a process
(Erlang process) running and managing a table of all modules currently
loaded in the VM. Each module can have 0, 1, or two versions of a module
loaded:

- 0: module isn't loaded
- 1: only a current version is loaded
- 2: a current version and an old version are loaded.

Let's focus on having one module loaded first. When you have one module
loaded, and then ask to load a new version of the module (done through
compiling, calling `l(Module)' from the shell, or through the `code'
module), the current version becomes the 'old' version, and the new one
becomes the 'current' version.

Every single process, by default, will keep calling the 'old' version
within local calls if they were in a local 'scope' for it already. When
you do a fully-qualified function call, you *always* get the newest
version. The mechanism is shown in http://learnyousomeerlang.com/designing-a-concurrent-application#hot-code-loving,
with an image explaining what I mean here.

Code reloading can be forced or controlled within a process by
restraining when a fully qualified call is made:

    -module(hotload).
    -export([server/1, upgrade/1]).
    
    server(State) ->
        receive
            update ->
                NewState = ?MODULE:upgrade(State),
                ?MODULE:server(NewState);  %% loop in the new version of the module
            SomeMessage ->
                %% do something here
                server(State)  %% stay in the same version no matter what.
        end.
    
    upgrade(OldState) ->
        %% transform and return the state here.

This requires understanding a bit of Erlang already.

Now what happens when we already have two modules loaded and we load a
third one? If we have processes still running code from the 'old'
module,  The Erlang VM more or less assumes it was an orphaned process
and brutally kills it. Otherwise (or once all the orphans are kileld),
the old copy is simply garbage collected.

For large scale systems using the OTP framework, this mechanism often
ends up not giving enough control to predictably upgrade to new versions
of the software. In such cases, the `sys' module can be used to suspend
individual processes and force them to adopt new code before resuming
them, and the entire thing can be wrapped in a concept called a `relup'
(which depends on you having build a `release').

The principles for relups and the `sys' modules are covered in
http://learnyousomeerlang.com/relups#the-hiccups-of-appups-and-relups if
you want a peek preview -- it's gonna be a while before relups are a
concern once you begin learning the language.

Regards,
Fred.