release handling
Peter H|gfeldt
peter@REDACTED
Wed Oct 11 19:32:36 CEST 2000
I wrote a utility called `uptest', the purpose of which is to test upgrade
and downgrade of one single application. The members of the OTP
development team use it internally. It is not open source (yet).
The attached document outlines its implementation, which reveales some
of the quirks of systools and release_handler, in particular the question
of creating the `initial' release.
/Peter
PS. Don't bother about missing *.gif files. DS.
-------------------------------------------------------------------------
Peter Högfeldt e-mail : peter@REDACTED
Open Telecom Platform Phone: : +46 (8) 727 57 58
Ericsson Utvecklings AB Mobile : +46 070-519 57 51
S-126 25 STOCKHOLM Fax: : +46 (8) 727 5775
Office address: Armborstvägen 1, Älvsjö
-------------- next part --------------
<HTML>
<HEAD>
<TITLE>Uptest Implementation</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF">
<IMG SRC="ERILOGO.GIF" ALT="Ericsson Blue Logotype" HEIGHT=38 WIDTH=187>
<TABLE CELLSPACING=0 CELLPADDING=2 BORDER=1 WIDTH="100%" >
<TR valign=top>
<TD width="50%" colspan=2><I><FONT SIZE=-2>Prepared<BR>
</FONT></I><B>Peter Högfeldt</B></TD>
<TD width="50%" colspan=3><I><FONT SIZE=-2>No.<BR>
</FONT></I><B><FONT SIZE=-1></FONT></B></TD>
</TR>
<TR valign=top>
<TD width="35%"><I><FONT SIZE=-2>Approved<BR>
</FONT></I><B><FONT SIZE=-1></FONT></B></TD>
<TD width="10%"><I><FONT SIZE=-2>Checked<BR>
</FONT></I><B><FONT SIZE=-1></FONT></B></TD>
<TD width="15%"><I><FONT SIZE=-2>Date<BR>
</FONT></I><B><FONT SIZE=-1>2000-02-09</FONT></B></TD>
<TD width="10%"><I><FONT SIZE=-2>Rev<BR>
</FONT></I><B><FONT SIZE=-1>PA2</FONT></B></TD>
<TD><I><FONT SIZE=-2>File<BR>
</FONT></I><B><FONT SIZE=-1>uptest_implementation.sgml</FONT></B></TD>
</TR>
</TABLE>
<CENTER>
<H1>Uptest Implementation</H1>
</CENTER>
<BLOCKQUOTE>
<A NAME="1"><!-- Empty --></A><H2>1 Introduction</H2><P>The purpose of the <CODE>uptest</CODE> utility is to simplify testing
of code replacement for Erlang applications.
<P>There are currently the following limitations:
<P><UL>
<LI>Erlang emulator version cannot be replaced,
</LI><BR>
<LI>only one application at a time can be tested.
</LI><BR>
</UL>
<A NAME="2"><!-- Empty --></A><H2>2 Work Phases</H2><P>Code replacement for an application can be tested by using the
<CODE>uptest</CODE> utility.There are two phases.
<P>The first phase consists of evaluating in any Erlang emulator
the following functions (in the order given).
<P><DL>
<DT><CODE>config()</CODE>
</DT>
<DD>Prompts the user for configuration parameters, for instance
the location of the Erlang emulator to use when testing code
replacement; the locations of the old and new versions of the
application to test.
</DD>
<DT><CODE>check()</CODE>
</DT>
<DD>Checks the consistency of contents of <CODE>ebin</CODE>
directories, <CODE>.app</CODE> files, and <CODE>.appup</CODE> file.
</DD>
<DT><CODE>setup()</CODE>
</DT>
<DD>Here two releases are prepared: <CODE>base</CODE> and <CODE>top</CODE>.
Boot scripts, <CODE>.rel</CODE> files, and a <CODE>relup</CODE> file are
created, and emulator executables and application code is
put in a test directory (specified in <CODE>config()</CODE>).
</DD>
</DL>
<P>In the second phase the emulator in the test directory (created
by <CODE>setup()</CODE>) is started, including the application to test,
and the following two functions can be evaluated:
<P><DL>
<DT><CODE>upgrade()</CODE>
</DT>
<DD>Upgrades the application from the base version to the
top version.
</DD>
<DT><CODE>downgrade()</CODE>
</DT>
<DD>Downgrades the application from the top version to the
base version.
</DD>
</DL>
<P>Both <CODE>upgrade()</CODE> and <CODE>downgrade()</CODE> checks the
supervision tree of the application before and after code
replacement, as well as the versions of replaced modules.
<A NAME="3"><!-- Empty --></A><H2>3 Details</H2><P>
<A NAME="3.1"><!-- Empty --></A><H3>3.1 config</H3><P>The user is prompted for the following information:
<P><OL>
<LI>an Erlang/OTP root directory,
</LI><BR>
<LI>the version of the Erlang runtime system to use,
</LI><BR>
<LI>the location and version of the application to
upgrade from (the <CODE>base</CODE> version),
</LI><BR>
<LI>the location and version of the application to
upgrade to (the <CODE>top</CODE> version),
</LI><BR>
<LI>the location of the test directory where code
replacement work shall be done.
</LI><BR>
</OL>
<P>Other applications needed by the tested application are
(recursively) found, including also kernel, stdlib, sasl and
uptest (which is later made into an application of its own).
<P>All information is written to the file <CODE>uptest.conf</CODE> in
the current directory.
<P>The following is an <STRONG>example configuration</STRONG>, which we
will refer to later on.
<P><STRONG>1. </STRONG>The application to test is <CODE>pea</CODE>, with
versions 1.0 and 1.1, located in
<CODE>/home/nisse/pea_build/lib/pea-1.0</CODE> and
<CODE>/home/nisse/pea_build/lib/pea-1.1</CODE> respectivly.
<P><STRONG>2. </STRONG>The Erlang root directory is <CODE>/usr/local/otp/r6b</CODE>
from which the emulator of version <CODE>4.9.1</CODE> will be picked.
<P><STRONG>3. </STRONG>The test directory is <CODE>/home/nisse/pea_test/TEST</CODE>.
<P><STRONG>4. </STRONG>Other applications needed are <CODE>kernel-2.4.3</CODE>,
<CODE>stdlib-1.8.2</CODE>, <CODE>sasl-1.8.2</CODE> and <CODE>uptest-1.1</CODE>
(yes, uptest will included as an application in the releases
built in <CODE>setup</CODE> below).
<A NAME="3.2"><!-- Empty --></A><H3>3.2 check</H3><P>Here the following things are checked for both the base version
and the top version of the application:
<P><STRONG>1. </STRONG>that the set of modules in the <CODE>ebin</CODE>
directory, and the set of modules in the <CODE>.app</CODE> do not
differ (if they do a warning is displayed),
<P><STRONG>2. </STRONG>that the set of added/removed modules according to
the <CODE>.app</CODE> file does not differ from that of the
<CODE>.appup</CODE> file,
<P><STRONG>3. </STRONG>by comparing module versions of the base and top
version of the application, detecting (i) those modules that are
changed according to the <CODE>.appup</CODE> file, but have the same
version, (ii) modules that have different versions, but are not
listed in the <CODE>.appup</CODE> file (if there are such modules a
warning is displayed).
<A NAME="3.3"><!-- Empty --></A><H3>3.3 setup</H3><P>This is the most elaborate function. We consider it in two
parts.
<P><STRONG>Note: </STRONG>Some intermediate files are created in the current
directory.
<A NAME="3.3.1"><!-- Empty --></A><H4>3.3.1 The base release</H4><P><STRONG>1. </STRONG>A <CODE>base.rel</CODE> is created in the current
directory:
<PRE> {release,{"TEST","1.0"},
{erts,"4.9.1"},
[{kernel,"2.4.3"},
{stdlib,"1.8.2"},
{sasl,"1.8.2"},
{uptest,"1.1"},
{pea,"1.0"}]}.
</PRE>
<P><STRONG>2. </STRONG>The <CODE>base.script</CODE> and <CODE>base.boot</CODE> files
are created in the current directory by a call
<PRE> systools:make_script("base", [{path, AppEbinDirs}]),
</PRE>
<P>where <CODE>AppEbinDirs</CODE> is a list of paths to all the
applications involved, so that the <CODE>.app</CODE> files can be
found and be incorporated in the boot script.
<P><STRONG>3. </STRONG>We also create a <CODE>plain.rel</CODE> file, and the
corresponding <CODE>plain.script</CODE> and <CODE>plain.boot</CODE> files
(in the current directory). The <CODE>plain.rel</CODE> file only
specifies the <CODE>kernel</CODE> and <CODE>stdlib</CODE> applications, so
that the <CODE>plain.boot</CODE> file can be used to start the system
without starting the application <CODE>pea</CODE>, which might be
needed if it has to be configured before it can be properly
started.
<P><STRONG>4. </STRONG>A tar file <CODE>base.tar.gz</CODE> is created in the
current directory by the call
<PRE> systools:make_tar("base", [{path, AppEbinDirs}, {erts, RootDir}]),
</PRE>
<P>where <CODE>AppEbinDirs</CODE> is as above, and <CODE>RootDir</CODE> is
the Erlang root directory (in the example configuration it is
<CODE>/usr/local/otp/r6b</CODE>). The tar file now contains the
following directories and files:
<PRE> lib/kernel-2.4.3/...
...
lib/pea-1.0/...
releases/base.rel
releases/1.0/start.boot
erts-4.9.1/...
</PRE>
<P>The <CODE>start.boot</CODE> is a copy of the <CODE>base.boot</CODE> file
in the current directory, and <CODE>base.rel</CODE> is the same as
<CODE>base.rel</CODE> in the current directory.
<P><STRONG>Note: </STRONG> Believe it or not, but
<CODE>systools:make_tar/2</CODE> tacitely incorporates into the
tar file the "system files" existing in the current
directory according to the following table (see the function
<CODE>add_system_files/4</CODE> in the module
<CODE>systools_make</CODE>). Here in our example configuration
case <CODE>RelFileName</CODE> = <CODE>"base"</CODE>, and <CODE>SVsn</CODE> =
<CODE>"1.0"</CODE>.
<P><CENTER>
<TABLE CELLSPACING=0 CELLPADDING=2 BORDER=1>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>File in current directory</STRONG>
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>Destination in tar file</STRONG>
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">RelFileName.boot
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">releases/SVsn/start.boot
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">relup
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">releases/SVsn/relup
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">sys.config
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">releases/SVsn/sys.config
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">RelFileName.rel
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">releases/RelFileName.rel
</TD>
</TR>
</TABLE>
</CENTER>
<P><STRONG>5. </STRONG>The tar file <CODE>base.tar.gz</CODE> in the current
directory is now extracted into the test directory by the call
<PRE> erl_tar:extract("base.tar.gz", [keep_old_files, {cwd, TestRoot},
compressed]),
</PRE>
<P>where <CODE>TestRoot</CODE> is the test directory (in our example is
is bound to <CODE>/home/nisse/pea_test/TEST</CODE>).
<P>In the test directory we now have the following directories:
<PRE> lib/kernel-2.4.3/...
...
lib/pea-1.0/...
releases/base.rel
releases/1.0/start.boot
erts-4.9.1/...
</PRE>
<P><STRONG>6. </STRONG>Within the test directory we create the file
<CODE>erts-4.9.1/bin/erl</CODE> by substituting the string
<CODE>%FINAL_ROOTDIR%</CODE> for <CODE>TestRoot</CODE> in the file
<CODE>erts-4.9.1/bin/erl.src</CODE>.
<P><STRONG>7. </STRONG>The directory <CODE>bin</CODE> is created within
the test directory, and the following files are copied within
the same directory:
<P><CENTER>
<TABLE CELLSPACING=0 CELLPADDING=2 BORDER=1>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>Source</STRONG>
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>Destination</STRONG>
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">erts-4.9.1/bin/erl
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/erl
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">erts-4.9.1/bin/erlc
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/erlc
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">erts-4.9.1/bin/epmd
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/epmd
</TD>
</TR>
</TABLE>
</CENTER>
<P><STRONG>8. </STRONG>From the current directory the following files are
copied into the test directory
<P><CENTER>
<TABLE CELLSPACING=0 CELLPADDING=2 BORDER=1>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>File in current directory</STRONG>
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE"><STRONG>Destination in test directory</STRONG>
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">base.script
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/base.script
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">base.boot
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/base.boot
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">plain.script
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/plain.script
</TD>
</TR>
<TR>
<TD ALIGN="LEFT" VALIGN="MIDDLE">plain.boot
</TD>
<TD ALIGN="LEFT" VALIGN="MIDDLE">bin/plain.boot
</TD>
</TR>
</TABLE>
</CENTER>
<P><STRONG>9. </STRONG>In the test directory the file
<CODE>releases/RELEASES</CODE> is created by the call
<PRE> release_handler:create_RELEASES(TestRoot, "base.rel"),
</PRE>
<P>where <CODE>TestRoot</CODE> is as above.
<P><STRONG>10. </STRONG>Within the test directory the file
<CODE>releases/start_erl.data</CODE> is created the contents of
which is the erts version and the release version - in our
example configuration one line containing
<PRE> "4.9.1 1.0"
</PRE>
<P><STRONG>11. </STRONG>In the current directory the file
<CODE>start_erl</CODE> is created, containing the following shell
script (this is for Unix only):
<PRE>
#!/bin/sh
#
read emu rel < %TEST%/releases/start_erl.data
exec %TEST%/bin/erl \
-boot %TEST%/releases/$rel/start \
-uptest workdir %WORKDIR% \
$*
</PRE>
<P>where <CODE>%TEST%</CODE> is substituted for the test directory,
and <CODE>%WORKDIR%</CODE> for the current directory.
<P>The purpose of this shell script is to start the system in
"code replacement mode".
<P><STRONG>12. </STRONG>In the current directory the file
<CODE>start_plain</CODE> is created, containing the following shell
script (this is for Unix only):
<PRE>
#!/bin/sh
#
exec %TEST%/bin/erl -boot %TEST%/bin/plain $*
</PRE>
<P>The purpose of this shell script is to be able to start a minimal
system in order to configure the application that is tested, before
it is really started.
<A NAME="3.3.2"><!-- Empty --></A><H4>3.3.2 The top release</H4><P>The <CODE>top</CODE> release is created similarly to the
<CODE>base</CODE> release. However, there are some differences.
XXX What differences?
<P><STRONG>1. </STRONG>A <CODE>top.rel</CODE> is created in the current
directory:
<PRE> {release,{"TEST","1.1"},
{erts,"4.9.1"},
[{kernel,"2.4.3"},
{stdlib,"1.8.2"},
{sasl,"1.8.2"},
{uptest,"1.1"},
{pea,"1.1"}]}.
</PRE>
<P><STRONG>2. </STRONG>The <CODE>top.script</CODE> and <CODE>top.boot</CODE> files
are created in the current directory by a call
<PRE> systools:make_script("top", [{path, AppEbinDirs}]),
</PRE>
<P>where <CODE>AppEbinDirs</CODE> is a list of paths to all the
applications involved, so that the <CODE>.app</CODE> files can be
found and be incorporated in the boot script.
<P><STRONG>3. </STRONG>A <CODE>relup</CODE> file is created in the current
directory by a call to
<PRE> systools:make_relup("top", ["base"], ["base"],
[{path, AppBaseEbinDirs ++ AppTopEbinDirs}]),
</PRE>
<P>where <CODE>AppBaseEbinDirs</CODE> and <CODE>AppTopEbinDirs</CODE> are
lists paths to <CODE>ebin</CODE> directories of the applications
in the <CODE>base</CODE> and <CODE>top</CODE> applications, respectively.
<P><STRONG>4. </STRONG>A tar file <CODE>top.tar.gz</CODE> is created in the
current directory by the call
<PRE> systools:make_tar("top", [{path, AppTopEbinDirs}]),
</PRE>
<P>where <CODE>AppTopEbinDirs</CODE> is as above, and <CODE>RootDir</CODE> is
the Erlang root directory (in the example configuration it is
<CODE>/usr/local/otp/r6b</CODE>). The tar file now contains the
following directories and files:
<PRE> lib/kernel-2.4.3/...
...
lib/pea-1.1/...
releases/top.rel
releases/1.0/relup
releases/1.0/start.boot
</PRE>
<P>The <CODE>start.boot</CODE> is a copy of the <CODE>top.boot</CODE> file
in the current directory, and <CODE>top.rel</CODE> is the same as
<CODE>top.rel</CODE> in the current directory.
<P>Note that the tar file does not contain an <CODE>erts</CODE>
directory.
<P><STRONG>5. </STRONG>The tar file <CODE>top.tar.gz</CODE> in the current
directory is copied to the <CODE>releases</CODE> directory within
the test directory.
<A NAME="3.4"><!-- Empty --></A><H3>3.4 upgrade</H3><P>The system is started by calling
<PRE> ./start_erl
</PRE>
<P>in the current directory.
<P><STRONG>1. </STRONG>The <CODE>top</CODE> release is unpacked by a call to
<PRE> release_handler:unpack_release("top").
</PRE>
<P>that returns a release version <CODE>Vsn</CODE> = <CODE>"1.0"</CODE>.
<P><STRONG>2. </STRONG>The <CODE>top</CODE> release is installed by a call to
<PRE> release_handler:install_release(Vsn).
</PRE>
<P>that performs code replacement for upgrading.
<P><STRONG>3. </STRONG>Next the release is made permanent by the call
<PRE> release_handler:make_permanent(Vsn)
</PRE>
<P>that changes the contents of <CODE>releases/RELEASES</CODE> and
<CODE>releases/start_erl.data</CODE>.
<A NAME="3.5"><!-- Empty --></A><H3>3.5 downgrade</H3><P>Similar to upgrade.
</BLOCKQUOTE>
<CENTER>
<HR>
<FONT SIZE=-1>
Copyright © 1991-2000
<A HREF="http://www.erlang.se">Ericsson Utvecklings AB</A><BR>
<!--#include virtual="/ssi/otp_footer.html"-->
</FONT>
</CENTER>
</BODY>
</HTML>
More information about the erlang-questions
mailing list