application upgrade FAQ (was gen_tcp:accept/{1,2} in passive , mode)

Thu Dec 14 19:16:49 CET 2000

On Thu, 14 Dec 2000, Sean Hinde wrote:

> Peter,
> 
> Any other top tips you might be able to share?
> 
> - Sean
> 

Sean,

please find attached the OTP internal FAQ regarding code replacement in
HTML format. It is rather a QNA (Questions Never Asked). Links are broken,
and you do not get the gif containing the Ericsson logo. 

It is provided "as is" under the usual conditions. Some of the tips might
be wrong.

/Peter 

-------------------------------------------------------------------------

Peter Högfeldt			e-mail  : peter@REDACTED
Open Telecom Platform		Phone:  : +46 (8) 727 57 58
Ericsson Utvecklings AB		Mobile	: +46  070-519 57 51
S-126 25 STOCKHOLM		Fax:	: +46 (8) 727 5775
Office address:			Armborstvägen 1, Älvsjö

-------------- next part --------------
<HTML>
<HEAD>
<TITLE>OTP Application Upgrade FAQ</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF">

<IMG SRC="ERILOGO.GIF" ALT="Ericsson Blue Logotype" HEIGHT=38 WIDTH=187>

<TABLE BORDER=1 WIDTH="100%" >
<TR valign=top>
<TD width="50%" colspan=2><I><FONT SIZE=-2>Prepared<BR>
</FONT></I><B>UAB/F/P P. Högfeldt</B></TD>

<TD width="50%" colspan=3><I><FONT SIZE=-2>No.<BR>
</FONT></I><B><FONT SIZE=-1>UAB/F-99:107</FONT></B></TD>
</TR>

<TR valign=top>
<TD width="35%">Approved 
UAB/F/P P. Högfeldt</TD>

<TD width="10%"><I><FONT SIZE=-2>Checked<BR>
</FONT></I><B><FONT SIZE=-1>PLH</FONT></B></TD>

<TD width="15%"><I><FONT SIZE=-2>Date<BR>
</FONT></I><B><FONT SIZE=-1>1999-08-12</FONT></B></TD>

<TD width="10%"><I><FONT SIZE=-2>Rev<BR>
</FONT></I><B><FONT SIZE=-1>PA6</FONT></B></TD>

<TD>File 
appup-faq.sgml</TD>
</TR>
</TABLE>
<CENTER>
<H1>OTP Application Upgrade FAQ</H1>
</CENTER>
<BLOCKQUOTE>
<A NAME="1"></A><H2>1 Frequently Asked Questions</H2>Q.1: I only updated the documentation. Shall the 
application version be incremented?

A: Yes.

Q.2: I only updated the documentation, and incremented
the application version. Is the new version to be used for
upgrade in run-time?

A: Probably not. However, the old <CODE>.appup</CODE> file
will do, if only it is updated with the new application version.

Q.3: How does the release handler find the processes
for which code change shall be performed?

A: By traversing the system supervision tree, and in
particular by reading the child specifications of
supervisors. See <A TARGET="_top" HREF="http://otp/product/onlinedocs/sasl/release_handling.html#4.5.1.5"> SASL User's Guide : Release Handling, Finding Processes</A>.

Q.4: What is a <CODE>.rel</CODE> file?

A: It is the release resource file that
contains a list of all application versions to be included in a
release (see <A TARGET="_top" HREF="http://otp/product/onlinedocs/sasl/rel.html"> rel(4)</A>). A <CODE>.rel</CODE> file is used as input to
<CODE>systools</CODE> (see <A TARGET="_top" HREF="http://otp/product/onlinedocs/sasl/systools.html"> systools(3)</A>) in order to generate <CODE>.script</CODE> and
<CODE>relup</CODE> files.

Q.5: What is a <CODE>.script</CODE> file?

A: It is a boot script file (see <A TARGET="_top" HREF="http://otp/product/onlinedocs/sasl/script.html"> script(4)</A>). A .script file is a text file that
describes how an Erlang system is started. It contains instructions
for loading code, and instructions for which applications and
processes to start.

Q.6: What is an <CODE>.appup</CODE> file?

A: It is an application upgrade file that
defines how an application is upgraded or downgraded (see <A TARGET="_top" HREF="http://otp/product/onlinedocs/sasl/appup.html"> appup(4)</A>). There is one <CODE>.appup</CODE> file for each
version of an application.

Q.7 : What is a <CODE>relup</CODE> file?

A: It is a release upgrade file that
specifies how to upgrade a node from old release versions to a
new version, and how to downgrade from the new version to old
release versions (see <A TARGET="_top" HREF="http://otp/product/onlinedocs/sasl/relup.html"> relup(4)</A>).

Q.8 : What is a <CODE>.boot</CODE> file?

A: It is the binary version of a <CODE>.script</CODE> file.

<P><STRONG>Q.9 : </STRONG>Which are the input and output files for
        <CODE>systools:make_script/1</CODE>?

A: The input files are a <CODE>.rel</CODE> file, and all
<CODE>.app</CODE> files of the applications listed in the <CODE>.rel</CODE>
file. The output file is a <CODE>.script</CODE> file.

<P><STRONG>Q.10 : </STRONG>Which are the input and output files for
        <CODE>systools:make_relup/3</CODE>?

A: The input files are one or several <CODE>.rel</CODE>
files, and all <CODE>.app</CODE> and <CODE>.appup</CODE> files of the
applications listed in the <CODE>.rel</CODE> files. The output file is
a <CODE>relup</CODE> file.

Q.11 : One process in my application is typically
hanging in a call to <CODE>gen_tcp:accept/1</CODE> waiting for a
connection request. How do I perform code change for that
process?

A: Let the implementation be such that another process
holds the listen socket and is linked to the process hanging in
<CODE>gen_tcp:accept/1</CODE>. Specify an <CODE>update</CODE> code upgrade 
instruction with a very short timeout, which will cause the
process to be killed almost immediately by the release handler,
before the new version of the call-back module for the process
is loaded. Then let the other process restart the process
calling <CODE>gen_tcp:accept/1</CODE>.

<P><STRONG>Q.12 : </STRONG>Can't I just put my processes under
        <CODE>kernel_safe_sup</CODE>?

A: No, the kernel supervisor <CODE>kernel_safe_sup</CODE> is
only to be used by certain server processes in the kernel. You
have to define a proper application, and start it through a call
to <CODE>application:start/1/2</CODE>.

<P><STRONG>Q.13 : </STRONG>Should you expect upgrades and downgrades to be
        symmetrical in the sense that if you have succeeded in
        implementing a really smooth upgrade, the downgrade will be
        smooth as well?

A: No. The focus is on upgrades that add functionality
or corrections in a <A HREF="#backwardcompatible">backward compatible</A>
manner. If for instance an API has been extended in an upgrade,
that extension will not be available after a downgrade. Still,
just before your application is downgraded, there might be other
applications pending on calls to the extension, which cannot be
serviced after the downgrade is in effect.

<P><STRONG>Q.14 : </STRONG>I have a port program in an application, the
        code of which has to be updated. It uses native operating
        system sockets, and I would like to change its code, without
        closing existing connections. Is there any generic support for
        this in OTP?

A: No. You should have implemented code change
functions in the old version of the port program, by having the
possibility of transferring state information of the port
program to the Erlang port controlling process, and then change
to a new version of the port program by ordering the old version
to <CODE>exec</CODE> (UNIX specific) the new version, and then let the
new version obtain the state from the controlling process. That
is not an easy thing to do.

<P><STRONG>Q.15: </STRONG>What does a `protocol change' mean?<A NAME="protocolchange"><!-- Empty --></A>

A: By the `protocol' between two processes, we mean the
syntax and semantics of all messages exchanged and recognized by
the processes. Hence a `protocol change' of this set of messages.
When the generic behaviours are used, the underlying messages 
are somewhat hidden, but roughly they are recognized forms of
<CODE>Request</CODE> in <CODE>gen_server:call/3</CODE> and the set of
replies etc.

Q.16: What is the meaning of `backward compatible'
<A NAME="backwardcompatible"></A>?

A: An new version of an implementation is backward
compatible if (i) the client API of the old version is a subset
or (possibly equal to) the client API of the new version, (ii)
the client-server protocol of the old version is a subset of
(possibly equal to) the client-server protocol of the new
version.

If a function that is a member of both the old and the new
version of a client API does not have the same set of
return values for all permitted values of the arguments
according to the old version of the API, then the new version is
not backward compatible.

Q.17 : In my application I have an API module
<CODE>myapp</CODE> that contains exported functions, that call
<A HREF="#clientfunction">client function</A>s in
call-back modules of the application. No process in my
application uses this module. Should it still be included in the
list of modules in child specifications.

A: No, but it should be listed in the <CODE>.app</CODE>
file, and if the module is changed, there must be an
<CODE>load_module</CODE> instruction for the module in the
<CODE>.appup</CODE> file.

<P><STRONG>Q.18: </STRONG>In my <CODE>gen_server</CODE> call-back module
<CODE>mymod</CODE> I have the following code:

<PRE>name(FirstName, SurName) ->
    gen_server:call(my_proc, {name, {FirstName, SurName}}).

...

handle_call({name, {SurName, FirstName}}, _From, State) -> 
    ...
    </PRE>
<P>where <CODE>FirstName</CODE> and <CODE>SurName</CODE> have been mixed up.
In correcting this bug, do I have to have a backward compatible
correction?
<P><STRONG>A: </STRONG>No, since it is a bug.

Q.19: In one of the <CODE>handle_call/3</CODE> clauses in 
a <CODE>gen_server</CODE> call-back module, I have a call to 
<CODE>lists:map/2</CODE>, the first argument of which is a fun that
is defined in the same module. Is that safe when changing code?

A: Yes.

Q.20: I have a server with <CODE>gen_server</CODE> behaviour.
Its task is to format reports according to one or serveral funs
that are installed by clients when they open the formatting
facility. Hence the funs becomes part of the state of the 
process. Is that safe?

A: No. The server will hold a reference to a fun not
defined in its own module. If a client makes a code change the
reference will be obsolete.

One way to avoid this situation is to disallow proper funs,
and instead let each client give a reference to an exported
function in the form <CODE>{M, F}</CODE>.

Q.21: I have an internal interface between two processes
in an application. Is it alright to make a change that is not 
backward compatible?

A: Yes, provided both clients and servers in the
application are updated to use the new version of the
interface. However, if the application is distributed, and if it
is likely that different versions will be running on a set of
nodes, parts of the internal interface must be backward
compatible. An example of this is a distributed application 
that is upgraded one node at a time.

Q.22: Is the protocol between <A HREF="#clientfunction">client function</A> and call-back
functions in a module just internal stuff, that I am free to
change as I like?

A: No, the protocol for a <A HREF="#clientfunction">client function</A> is not
internal if the client function is part of an external API to be
used by other applications.

Q.23: We have <CODE>.appup</CODE> files. Why do we not have
<CODE>.appdown</CODE> files as well?

A: Well, an <CODE>.appup</CODE> file contains instructions both
for upgrades and downgrades. Maybe a better name would have been
<CODE>.appchange</CODE>.

Q.24: I have found a bug in the <CODE>gen_server</CODE> module,
and I know how to correct it. How do perform smooth code change
for the new version?

A: Load the new version, do suspend and resume on all
processes with gen_server behaviour, and remove the old version.

Q.25: I have to make a change to the external interface
of my application, and that change is certainly not backward
compatible. My application is used by other OTP applications, 
and also by other applications using Erlang/OTP. How is an
upgrade performed?

A: You have to have the permission of product management
to introduce such a change.

An upgrade is specified as follows. All applications using your
application through its external interface, have to state in
their upgrade scripts that they depend on your application.
Specifically, each updated module belonging to the other
applications, has to have an <CODE>update</CODE> instruction with a
dependency list containing the name of your external interface
module, and all call-back modules of your application
that are referenced from your interface module.

You must specify the dependency list in the release notes of
your application.

Q.26 What is a <CODE>code_change</CODE> function and how it
is used?

A: The <CODE>code_change</CODE> function is a call-back 
function in all <CODE>gen_*</CODE> behaviours. In a <CODE>code_change</CODE>
function a process typically converts its state from the old to
a new form compatible with the new code in case of an upgrade,
and vice versa at downgrade. Furthermore, if applicable, it
transforms storage formats and data base tables.

For examples of usage, see <A TARGET="_top" HREF="http://otp/product/onlinedocs/sasl/release_handling.html#4.7"> SASL User's Guide : Release Handling Examples</A>.

Q.27 What do I have consider when transforming large
tables in Mnesia?

A: Try to implement lazy transformation.

Q.28 What is a <CODE>confic_change</CODE> function?

A: When an application changes its configuration from
one release to another, the <CODE>config_change</CODE> function is
called. See <A TARGET="_top" HREF="http://otp/product/onlinedocs/sasl/application.html"> application(3)</A>.

Q.29 I have a call-back module where I have defined 
<CODE>handle_call/3</CODE> function. Should I have a last "catch all"
clause? Example:

<PRE>handle_call(Req, _From, State) ->
    %% Request not recognized
    {reply, {error, Req}, State}.
      </PRE>
<P><STRONG>A: </STRONG>Yes. If you extend the protocol from one version to
new version by adding a <CODE>handle_call</CODE> clause, then a downgrade
from the new version to the previous version might result in that
the old version does not recognize a protocol message that was
formed just before the downgrade was done by a call to a <A HREF="#clientfunction">client function</A> in the new version.

Q.30: <A NAME="clientfunction"></A>
What do you mean my a `client function'?

A: In the example of Q.18 the <CODE>name</CODE> is
a client function (and <CODE>handle_call</CODE> is a call-back function
as defined in the <CODE>gen_server</CODE> behaviour).

<P><STRONG>Q.31: </STRONG>

<P><STRONG>A: </STRONG>

</BLOCKQUOTE>
<CENTER>
<HR>
<FONT SIZE=-1>
Copyright © 1991-1999
<A HREF="http://www.erlang.se">Ericsson Utvecklings AB</A><BR>
<!--#include virtual="/ssi/otp_footer.html"-->
</FONT>
</CENTER>
</BODY>
</HTML>