Deployment ( digression from: Longstanding issues: ...)

[Ulf Wiger (AL/EAB)]

Mikael Karlsson wrote:
> 
> I am still a bit lost though, with the concept of one boot 
> file per application in the priv dir.

The concept I wanted to try out with 'builder' was to 
build a system incrementally, one application at a time.
Basically, you could see it as building a unit test 
environment for each application. For each application,
there's a boot script that fires up the applications 
needed by that particular application. The higher up
you get, the more of the final system will be started.
And you're always using the OTP scripts, so the unit 
test will be very similar to what happens in the final
system.

Unfortunately, builder is currently not very good at
building a 'system', where the boot script is _not_
in the priv/ directory of some application. It was 
supposed to work, but apparently, I didn't test it. :-/

> My use case - scenario is this.
> Precondition:
> - a distributed erlang environment on several machines.
> - Erlang OTP installed in /usr/local/lib/erlang.
> - I "third party" application installed in /usr/local/yaws ( 
> .erlang is
>   used to add  this to the path)
> - Starting erlang from an ordinary user /home/usr
> - export ERL_BASE=/home/usr/erl_base
> - All new apps to go into $ERL_BASE/lib, on all nodes.
> 
> Can I, for all nodes in one go:
> 1. Make a relase package with one new application app1-vsn11 
> that goes into $ERL_BASE/lib 2. Make a release upgrade, 
> upgrading to app1-vsn12 and add a new app2-vsn21 3. Make a 
> release upgrade, adding app3-vsn31 and keep the old ones as is.
> 4. Make a release upgrade removing app2-vsn21, keeping 
> everything else.
> 
> And 1-4 without having to deal with all OTP application versions.

I'm not sure what kind of upgrade you envision that does
not deal with the application versions. Or do you mean 
that you don't want to mess with putting in versions in 
all the different text files needed for systools?
If so, that can be done ('builder' was supposed to do that 
too, but doesn't yet...) What builder does do is allow you 
either specify exactly which version of a particular app
that you want to use, or have 'builder' simply select the
most recent one.

Upgrade is a b**** if you wan to do it smoothly. I think the
Erlang/OTP code change support is mainly suited to fairly 
simple upgrades -- but that's fine, because we do a lot of 
those (error corrections, and 'interactive debugging'). In
a large system with up to a hundred or so applications and
thousands of processes, it is difficult to keep track of 
what actually happens when all processes are supposed to 
do their own code change... or rather, what tends to mess
things up royally is restructuring of process hierarchies,
renaming, deleting and moving ets tables, etc. And in a 
large system, all these things _will_ happen during a 
major upgrade. Also, equivalence test becomes extremely
difficult -- how do you now that the "organically"
upgraded system is equivalent to one that's been 
installed and initialized from scratch?

There are other ways to do it: Mnesia nowadays is quite
robust when it comes to bootstrapping a new schema from
another node, and rather than transform_tables(), which
happens within a (potentially huge) transaction, one can
use traverse_backup() and transform both schema and data
off-line. Then, one can restart the system. If it's a 
redundant system, the nodes have to be separated while 
upgrading the mnesia schema (assuming the schema needs
changing). This can be done e.g. with cookies. If there
are version differences in the application-level 
message-passing between nodes, this has to be taken 
care of by the applications, and no pre-package solution
exists for that, AFAIK. 

I don't know of anyone who actually does upgrades like
this, but it should work(tm).  (:


/Ulf W