release handling

Peter H|gfeldt peter@REDACTED
Wed Oct 11 19:32:36 CEST 2000


I wrote a utility called `uptest', the purpose of which is to test upgrade
and downgrade of one single application. The members of the OTP
development team use it internally. It is not open source (yet).

The attached document outlines its implementation, which reveales some
of the quirks of systools and release_handler, in particular the question
of creating the `initial' release.

/Peter    

PS. Don't bother about missing *.gif files. DS.

-------------------------------------------------------------------------
Peter Högfeldt			e-mail  : peter@REDACTED
Open Telecom Platform		Phone:  : +46 (8) 727 57 58
Ericsson Utvecklings AB		Mobile	: +46  070-519 57 51
S-126 25 STOCKHOLM		Fax:	: +46 (8) 727 5775
Office address:			Armborstvägen 1, Älvsjö


-------------- next part --------------
<HTML>
<HEAD>
<TITLE>Uptest Implementation</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF">

<IMG SRC="ERILOGO.GIF" ALT="Ericsson Blue Logotype" HEIGHT=38 WIDTH=187>

<TABLE CELLSPACING=0 CELLPADDING=2 BORDER=1 WIDTH="100%" >
<TR valign=top>
<TD width="50%" colspan=2><I><FONT SIZE=-2>Prepared<BR>
</FONT></I><B>Peter Högfeldt</B></TD>

<TD width="50%" colspan=3><I><FONT SIZE=-2>No.<BR>
</FONT></I><B><FONT SIZE=-1></FONT></B></TD>
</TR>

<TR valign=top>
<TD width="35%"><I><FONT SIZE=-2>Approved<BR>
</FONT></I><B><FONT SIZE=-1></FONT></B></TD>

<TD width="10%"><I><FONT SIZE=-2>Checked<BR>
</FONT></I><B><FONT SIZE=-1></FONT></B></TD>

<TD width="15%"><I><FONT SIZE=-2>Date<BR>
</FONT></I><B><FONT SIZE=-1>2000-02-09</FONT></B></TD>

<TD width="10%"><I><FONT SIZE=-2>Rev<BR>
</FONT></I><B><FONT SIZE=-1>PA2</FONT></B></TD>

<TD><I><FONT SIZE=-2>File<BR>
</FONT></I><B><FONT SIZE=-1>uptest_implementation.sgml</FONT></B></TD>
</TR>
</TABLE>
<CENTER>
<H1>Uptest Implementation</H1>
</CENTER>
<BLOCKQUOTE>
<A NAME="1"><!-- Empty --></A><H2>1 Introduction</H2><P>The purpose of the <CODE>uptest</CODE> utility is to simplify testing
of code replacement for Erlang applications.

<P>There are currently the following limitations:

<P><UL>
<LI>Erlang emulator version cannot be replaced,
</LI><BR>
<LI>only one application at a time can be tested.
</LI><BR>
</UL>
<A NAME="2"><!-- Empty --></A><H2>2 Work Phases</H2><P>Code replacement for an application can be tested by using the
<CODE>uptest</CODE> utility.There are two phases. 

<P>The first phase consists of evaluating in any Erlang emulator
the following functions (in the order given).

<P><DL>
<DT><CODE>config()</CODE>

</DT>
<DD>Prompts the user for configuration parameters, for instance
        the location of the Erlang emulator to use when testing code
        replacement; the locations of the old and new versions of the
        application to test.

</DD>
<DT><CODE>check()</CODE> 

</DT>
<DD>Checks the consistency of contents of <CODE>ebin</CODE>
        directories, <CODE>.app</CODE> files, and <CODE>.appup</CODE> file.

</DD>
<DT><CODE>setup()</CODE>

</DT>
<DD>Here two releases are prepared: <CODE>base</CODE> and <CODE>top</CODE>.
        Boot scripts, <CODE>.rel</CODE> files, and a <CODE>relup</CODE> file are
        created, and emulator executables and application code is
        put in a test directory (specified in <CODE>config()</CODE>).
        
</DD>
</DL>
<P>In the second phase the emulator in the test directory (created
by <CODE>setup()</CODE>) is started, including the application to test,
and the following two functions can be evaluated:

<P><DL>
<DT><CODE>upgrade()</CODE>

</DT>
<DD>Upgrades the application from the base version to the 
        top version.

</DD>
<DT><CODE>downgrade()</CODE>

</DT>
<DD>Downgrades the application from the top version to the
        base version.

</DD>
</DL>
<P>Both <CODE>upgrade()</CODE> and <CODE>downgrade()</CODE> checks the
supervision tree of the application before and after code
replacement, as well as the versions of replaced modules.

<A NAME="3"><!-- Empty --></A><H2>3 Details</H2><P>
<A NAME="3.1"><!-- Empty --></A><H3>3.1 config</H3><P>The user is prompted for the following information:

<P><OL>
<LI>an Erlang/OTP root directory,
        </LI><BR>
<LI>the version of the Erlang runtime system to use,
        </LI><BR>
<LI>the location and version of the application to
         upgrade from (the <CODE>base</CODE> version),
        </LI><BR>
<LI>the location and version of the application to
         upgrade to (the <CODE>top</CODE> version),
        </LI><BR>
<LI>the location of the test directory where code
         replacement work shall be done.
</LI><BR>
</OL>
<P>Other applications needed by the tested application are
        (recursively) found, including also kernel, stdlib, sasl and
        uptest (which is later made into an application of its own).

<P>All information is written to the file <CODE>uptest.conf</CODE> in
        the current directory.

<P>The following is an <STRONG>example configuration</STRONG>, which we
        will refer to later on.

<P><STRONG>1. </STRONG>The application to test is <CODE>pea</CODE>, with
        versions 1.0 and 1.1, located in
        <CODE>/home/nisse/pea_build/lib/pea-1.0</CODE> and
        <CODE>/home/nisse/pea_build/lib/pea-1.1</CODE> respectivly.

<P><STRONG>2. </STRONG>The Erlang root directory is <CODE>/usr/local/otp/r6b</CODE>
        from which the emulator of version <CODE>4.9.1</CODE> will be picked.

<P><STRONG>3. </STRONG>The test directory is <CODE>/home/nisse/pea_test/TEST</CODE>.

<P><STRONG>4. </STRONG>Other applications needed are <CODE>kernel-2.4.3</CODE>,
        <CODE>stdlib-1.8.2</CODE>, <CODE>sasl-1.8.2</CODE> and <CODE>uptest-1.1</CODE>
        (yes, uptest will included as an application in the releases
        built in <CODE>setup</CODE> below).

<A NAME="3.2"><!-- Empty --></A><H3>3.2 check</H3><P>Here the following things are checked for both the base version
        and the top version of the application:

<P><STRONG>1. </STRONG>that the set of modules in the <CODE>ebin</CODE>
        directory, and the set of modules in the <CODE>.app</CODE> do not
        differ (if they do a warning is displayed),

<P><STRONG>2. </STRONG>that the set of added/removed modules according to
        the <CODE>.app</CODE> file does not differ from that of the
        <CODE>.appup</CODE> file,

<P><STRONG>3. </STRONG>by comparing module versions of the base and top
        version of the application, detecting (i) those modules that are
        changed according to the <CODE>.appup</CODE> file, but have the same
        version, (ii) modules that have different versions, but are not
        listed in the <CODE>.appup</CODE> file (if there are such modules a
        warning is displayed).

<A NAME="3.3"><!-- Empty --></A><H3>3.3 setup</H3><P>This is the most elaborate function. We consider it in two
        parts. 

<P><STRONG>Note: </STRONG>Some intermediate files are created in the current
        directory.
        
<A NAME="3.3.1"><!-- Empty --></A><H4>3.3.1 The base release</H4><P><STRONG>1. </STRONG>A <CODE>base.rel</CODE> is created in the current
         directory:

        <PRE>     {release,{"TEST","1.0"},
          {erts,"4.9.1"},
          [{kernel,"2.4.3"},
          {stdlib,"1.8.2"},
          {sasl,"1.8.2"},
          {uptest,"1.1"},
          {pea,"1.0"}]}.

        </PRE>
<P><STRONG>2. </STRONG>The <CODE>base.script</CODE> and <CODE>base.boot</CODE> files
         are created in the current directory by a call
        <PRE>     systools:make_script("base", [{path, AppEbinDirs}]),
        </PRE>
<P>where <CODE>AppEbinDirs</CODE> is a list of paths to all the
         applications involved, so that the <CODE>.app</CODE> files can be
         found and be incorporated in the boot script.

        <P><STRONG>3. </STRONG>We also create a <CODE>plain.rel</CODE> file, and the
         corresponding <CODE>plain.script</CODE> and <CODE>plain.boot</CODE> files
         (in the current directory). The <CODE>plain.rel</CODE> file only
         specifies the <CODE>kernel</CODE> and <CODE>stdlib</CODE> applications, so
         that the <CODE>plain.boot</CODE> file can be used to start the system
         without starting the application <CODE>pea</CODE>, which might be
         needed if it has to be configured before it can be properly
         started.

        <P><STRONG>4. </STRONG>A tar file <CODE>base.tar.gz</CODE> is created in the
         current directory by the call 
        <PRE>     systools:make_tar("base", [{path, AppEbinDirs}, {erts, RootDir}]), 
        </PRE>
<P>where <CODE>AppEbinDirs</CODE> is as above, and <CODE>RootDir</CODE> is
         the Erlang root directory (in the example configuration it is
         <CODE>/usr/local/otp/r6b</CODE>). The tar file now contains the
         following directories and files:

        <PRE>     lib/kernel-2.4.3/...
          ...
          lib/pea-1.0/...
          releases/base.rel
          releases/1.0/start.boot
          erts-4.9.1/...
        </PRE>
<P>The <CODE>start.boot</CODE> is a copy of the <CODE>base.boot</CODE> file
         in the current directory, and <CODE>base.rel</CODE> is the same as
         <CODE>base.rel</CODE> in the current directory.

        <P><STRONG>Note: </STRONG> Believe it or not, but
         <CODE>systools:make_tar/2</CODE> tacitely incorporates into the
         tar file the "system files" existing in the current
         directory according to the following table (see the function
         <CODE>add_system_files/4</CODE> in the module
         <CODE>systools_make</CODE>). Here in our example configuration
         case <CODE>RelFileName</CODE> = <CODE>"base"</CODE>, and <CODE>SVsn</CODE> =
         <CODE>"1.0"</CODE>.

        <P><CENTER>
<TABLE CELLSPACING=0 CELLPADDING=2 BORDER=1>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>File in current directory</STRONG>
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>Destination in tar file</STRONG>
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">RelFileName.boot
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">releases/SVsn/start.boot
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">relup
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">releases/SVsn/relup
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">sys.config
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">releases/SVsn/sys.config
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">RelFileName.rel
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">releases/RelFileName.rel
         </TD>
</TR>
</TABLE>
</CENTER>
<P><STRONG>5. </STRONG>The tar file <CODE>base.tar.gz</CODE> in the current
         directory is now extracted into the test directory by the call
         
        <PRE>     erl_tar:extract("base.tar.gz", [keep_old_files, {cwd, TestRoot}, 
          compressed]),
        </PRE>
<P>where <CODE>TestRoot</CODE> is the test directory (in our example is
         is bound to <CODE>/home/nisse/pea_test/TEST</CODE>).

        <P>In the test directory we now have the following directories:

        <PRE>     lib/kernel-2.4.3/...
          ...
          lib/pea-1.0/...
          releases/base.rel
          releases/1.0/start.boot
          erts-4.9.1/...
        </PRE>
<P><STRONG>6. </STRONG>Within the test directory we create the file
         <CODE>erts-4.9.1/bin/erl</CODE> by substituting the string
         <CODE>%FINAL_ROOTDIR%</CODE> for <CODE>TestRoot</CODE> in the file
         <CODE>erts-4.9.1/bin/erl.src</CODE>.

        <P><STRONG>7. </STRONG>The directory <CODE>bin</CODE> is created within
         the test directory, and the following files are copied within
         the same directory:

        <P><CENTER>
<TABLE CELLSPACING=0 CELLPADDING=2 BORDER=1>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>Source</STRONG>
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>Destination</STRONG>
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">erts-4.9.1/bin/erl
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/erl
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">erts-4.9.1/bin/erlc
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/erlc
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">erts-4.9.1/bin/epmd
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/epmd
         </TD>
</TR>
</TABLE>
</CENTER>
<P><STRONG>8. </STRONG>From the current directory the following files are
         copied into the test directory

        <P><CENTER>
<TABLE CELLSPACING=0 CELLPADDING=2 BORDER=1>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>File in current directory</STRONG>
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>Destination in test directory</STRONG>
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">base.script
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/base.script
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">base.boot
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/base.boot
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">plain.script
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/plain.script
         </TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">plain.boot
         </TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/plain.boot
         </TD>
</TR>
</TABLE>
</CENTER>
<P><STRONG>9. </STRONG>In the test directory the file
         <CODE>releases/RELEASES</CODE> is created by the call

        <PRE>     release_handler:create_RELEASES(TestRoot, "base.rel"),
        </PRE>
<P>where <CODE>TestRoot</CODE> is as above.

         
        <P><STRONG>10. </STRONG>Within the test directory the file
         <CODE>releases/start_erl.data</CODE> is created the contents of
         which is the erts version and the release version - in our
         example configuration one line containing 

        <PRE>     "4.9.1 1.0"
        </PRE>
<P><STRONG>11. </STRONG>In the current directory the file
         <CODE>start_erl</CODE> is created, containing the following shell
         script (this is for Unix only):

        <PRE>
          #!/bin/sh
          #
          read emu rel < %TEST%/releases/start_erl.data
          exec %TEST%/bin/erl \
          -boot %TEST%/releases/$rel/start \
          -uptest workdir %WORKDIR% \
          $*
        </PRE>
<P>where <CODE>%TEST%</CODE> is substituted for the test directory,
         and <CODE>%WORKDIR%</CODE> for the current directory.

        <P>The purpose of this shell script is to start the system in
         "code replacement mode".

        <P><STRONG>12. </STRONG>In the current directory the file
         <CODE>start_plain</CODE> is created, containing the following shell
         script (this is for Unix only):

        <PRE>
          #!/bin/sh
          #
          exec %TEST%/bin/erl -boot %TEST%/bin/plain $*
        </PRE>
<P>The purpose of this shell script is to be able to start a minimal
         system in order to configure the application that is tested, before
         it is really started.

<A NAME="3.3.2"><!-- Empty --></A><H4>3.3.2 The top release</H4><P>The <CODE>top</CODE> release is created similarly to the
         <CODE>base</CODE> release. However, there are some differences. 
         XXX What differences?

        <P><STRONG>1. </STRONG>A <CODE>top.rel</CODE> is created in the current
         directory:

        <PRE>     {release,{"TEST","1.1"},
          {erts,"4.9.1"},
          [{kernel,"2.4.3"},
          {stdlib,"1.8.2"},
          {sasl,"1.8.2"},
          {uptest,"1.1"},
          {pea,"1.1"}]}.

        </PRE>
<P><STRONG>2. </STRONG>The <CODE>top.script</CODE> and <CODE>top.boot</CODE> files
         are created in the current directory by a call
        <PRE>     systools:make_script("top", [{path, AppEbinDirs}]),
        </PRE>
<P>where <CODE>AppEbinDirs</CODE> is a list of paths to all the
         applications involved, so that the <CODE>.app</CODE> files can be
         found and be incorporated in the boot script.

        <P><STRONG>3. </STRONG>A <CODE>relup</CODE> file is created in the current
         directory by a call to

        <PRE>     systools:make_relup("top", ["base"], ["base"], 
                              [{path, AppBaseEbinDirs ++ AppTopEbinDirs}]),
        </PRE>
<P>where <CODE>AppBaseEbinDirs</CODE> and <CODE>AppTopEbinDirs</CODE> are
         lists paths to <CODE>ebin</CODE> directories of the applications
         in the <CODE>base</CODE> and <CODE>top</CODE> applications, respectively.

        <P><STRONG>4. </STRONG>A tar file <CODE>top.tar.gz</CODE> is created in the
         current directory by the call 
        <PRE>     systools:make_tar("top", [{path, AppTopEbinDirs}]), 
        </PRE>
<P>where <CODE>AppTopEbinDirs</CODE> is as above, and <CODE>RootDir</CODE> is
         the Erlang root directory (in the example configuration it is
         <CODE>/usr/local/otp/r6b</CODE>). The tar file now contains the
         following directories and files:

        <PRE>     lib/kernel-2.4.3/...
          ...
          lib/pea-1.1/...
          releases/top.rel
          releases/1.0/relup
          releases/1.0/start.boot
        </PRE>
<P>The <CODE>start.boot</CODE> is a copy of the <CODE>top.boot</CODE> file
         in the current directory, and <CODE>top.rel</CODE> is the same as
         <CODE>top.rel</CODE> in the current directory.

        <P>Note that the tar file does not contain an <CODE>erts</CODE>
         directory.

        <P><STRONG>5. </STRONG>The tar file <CODE>top.tar.gz</CODE> in the current
         directory is copied to the <CODE>releases</CODE> directory within
         the test directory.

<A NAME="3.4"><!-- Empty --></A><H3>3.4 upgrade</H3><P>The system is started by calling

<PRE>   ./start_erl
      </PRE>
<P>in the current directory.
        

<P><STRONG>1. </STRONG>The <CODE>top</CODE> release is unpacked by a call to 

<PRE>   release_handler:unpack_release("top").
      </PRE>
<P>that returns a release version <CODE>Vsn</CODE> = <CODE>"1.0"</CODE>.

<P><STRONG>2. </STRONG>The <CODE>top</CODE> release is installed by a call to
        
<PRE>   release_handler:install_release(Vsn).
      </PRE>
<P>that performs code replacement for upgrading.

<P><STRONG>3. </STRONG>Next the release is made permanent by the call

<PRE>   release_handler:make_permanent(Vsn)
      </PRE>
<P>that changes the contents of <CODE>releases/RELEASES</CODE> and
        <CODE>releases/start_erl.data</CODE>.
<A NAME="3.5"><!-- Empty --></A><H3>3.5 downgrade</H3><P>Similar to upgrade.

</BLOCKQUOTE>
<CENTER>
<HR>
<FONT SIZE=-1>
Copyright © 1991-2000
<A HREF="http://www.erlang.se">Ericsson Utvecklings AB</A><BR>
<!--#include virtual="/ssi/otp_footer.html"-->
</FONT>
</CENTER>
</BODY>
</HTML>


More information about the erlang-questions mailing list