1 cover

1.1 Introduction

The module cover provides a set of functions for coverage analysis of Erlang programs, counting how many times each executable line is executed.

Coverage analysis can be used to verify test cases, making sure all relevant code is covered, and may be helpful when looking for bottlenecks in the code.

1.2 Getting Started With Cover

Example

Assume that a test case for the following program should be verified:

-module(channel).
-behaviour(gen_server).

-export([start_link/0,stop/0]).
-export([alloc/0,free/1]). % client interface
-export([init/1,handle_call/3,terminate/2]). % callback functions

start_link() ->
    gen_server:start_link({local,channel},channel,[],[]).

stop() ->
    gen_server:call(channel,stop).

%%%-Client interface functions-------------------------------------------

alloc() ->
    gen_server:call(channel,alloc).

free(Channel) ->
    gen_server:call(channel,{free,Channel}).

%%%-gen_server callback functions----------------------------------------

init(_Arg) ->
    {ok,channels()}.

handle_call(stop,Client,Channels) ->
    {stop,normal,ok,Channels};

handle_call(alloc,Client,Channels) ->
    {Ch,Channels2} = alloc(Channels),
    {reply,{ok,Ch},Channels2};

handle_call({free,Channel},Client,Channels) ->
    Channels2 = free(Channel,Channels),
    {reply,ok,Channels2}.

terminate(_Reason,Channels) ->
    ok.

%%%-Internal functions---------------------------------------------------

channels() ->
    [ch1,ch2,ch3].

alloc([Channel|Channels]) ->
    {Channel,Channels};
alloc([]) ->
    false.

free(Channel,Channels) ->
    [Channel|Channels].

The test case is implemented as follows:

-module(test).
-export([s/0]).

s() ->
    {ok,Pid} = channel:start_link(),
    {ok,Ch1} = channel:alloc(),
    ok = channel:free(Ch1),
    ok = channel:stop().

Preparation

First of all, Cover must be started. This spawns a process which owns the Cover database where all coverage data will be stored.

1> cover:start().
{ok,<0.30.0>}

To include other nodes in the coverage analysis, use start/1. All cover compiled modules will then be loaded on all nodes, and data from all nodes will be summed up when analysing. For simplicity this example only involves the current node.

Before any analysis can take place, the involved modules must be Cover compiled. This means that some extra information is added to the module before it is compiled into a binary which then is loaded. The source file of the module is not affected and no .beam file is created.

2> cover:compile_module(channel).
{ok,channel}

Each time a function in the Cover compiled module channel is called, information about the call will be added to the Cover database. Run the test case:

3> test:s().
ok

Cover analysis is performed by examining the contents of the Cover database. The output is determined by two parameters, Level and Analysis. Analysis is either coverage or calls and determines the type of the analysis. Level is either module, function, clause, or line and determines the level of the analysis.

Coverage Analysis

Analysis of type coverage is used to find out how much of the code has been executed and how much has not been executed. Coverage is represented by a tuple {Cov,NotCov}, where Cov is the number of executable lines that have been executed at least once and NotCov is the number of executable lines that have not been executed.

If the analysis is made on module level, the result is given for the entire module as a tuple {Module,{Cov,NotCov}}:

4> cover:analyse(channel,coverage,module).
{ok,{channel,{14,1}}}

For channel, the result shows that 14 lines in the module are covered but one line is not covered.

If the analysis is made on function level, the result is given as a list of tuples {Function,{Cov,NotCov}}, one for each function in the module. A function is specified by its module name, function name and arity:

5> cover:analyse(channel,coverage,function).
{ok,[{{channel,start_link,0},{1,0}},
     {{channel,stop,0},{1,0}},
     {{channel,alloc,0},{1,0}},
     {{channel,free,1},{1,0}},
     {{channel,init,1},{1,0}},
     {{channel,handle_call,3},{5,0}},
     {{channel,terminate,2},{1,0}},
     {{channel,channels,0},{1,0}},
     {{channel,alloc,1},{1,1}},
     {{channel,free,2},{1,0}}]}

For channel, the result shows that the uncovered line is in the function channel:alloc/1.

If the analysis is made on clause level, the result is given as a list of tuples {Clause,{Cov,NotCov}}, one for each function clause in the module. A clause is specified by its module name, function name, arity and position within the function definition:

6> cover:analyse(channel,coverage,clause).
{ok,[{{channel,start_link,0,1},{1,0}},
     {{channel,stop,0,1},{1,0}},
     {{channel,alloc,0,1},{1,0}},
     {{channel,free,1,1},{1,0}},
     {{channel,init,1,1},{1,0}},
     {{channel,handle_call,3,1},{1,0}},
     {{channel,handle_call,3,2},{2,0}},
     {{channel,handle_call,3,3},{2,0}},
     {{channel,terminate,2,1},{1,0}},
     {{channel,channels,0,1},{1,0}},
     {{channel,alloc,1,1},{1,0}},
     {{channel,alloc,1,2},{0,1}},
     {{channel,free,2,1},{1,0}}]}

For channel, the result shows that the uncovered line is in the second clause of channel:alloc/1.

Finally, if the analysis is made on line level, the result is given as a list of tuples {Line,{Cov,NotCov}}, one for each executable line in the source code. A line is specified by its module name and line number.

7> cover:analyse(channel,coverage,line).
{ok,[{{channel,9},{1,0}},
     {{channel,12},{1,0}},
     {{channel,17},{1,0}},
     {{channel,20},{1,0}},
     {{channel,25},{1,0}},
     {{channel,28},{1,0}},
     {{channel,31},{1,0}},
     {{channel,32},{1,0}},
     {{channel,35},{1,0}},
     {{channel,36},{1,0}},
     {{channel,39},{1,0}},
     {{channel,44},{1,0}},
     {{channel,47},{1,0}},
     {{channel,49},{0,1}},
     {{channel,52},{1,0}}]}

For channel, the result shows that the uncovered line is line number 49.

Call Statistics

Analysis of type calls is used to find out how many times something has been called and is represented by an integer Calls.

If the analysis is made on module level, the result is given as a tuple {Module,Calls}. Here Calls is the total number of calls to functions in the module:

8> cover:analyse(channel,calls,module).
{ok,{channel,12}}

For channel, the result shows that a total of twelve calls have been made to functions in the module.

If the analysis is made on function level, the result is given as a list of tuples {Function,Calls}. Here Calls is the number of calls to each function:

9> cover:analyse(channel,calls,function).
{ok,[{{channel,start_link,0},1},
     {{channel,stop,0},1},
     {{channel,alloc,0},1},
     {{channel,free,1},1},
     {{channel,init,1},1},
     {{channel,handle_call,3},3},
     {{channel,terminate,2},1},
     {{channel,channels,0},1},
     {{channel,alloc,1},1},
     {{channel,free,2},1}]}

For channel, the result shows that handle_call/3 is the most called function in the module (three calls). All other functions have been called once.

If the analysis is made on clause level, the result is given as a list of tuples {Clause,Calls}. Here Calls is the number of calls to each function clause:

10> cover:analyse(channel,calls,clause).
{ok,[{{channel,start_link,0,1},1},
     {{channel,stop,0,1},1},
     {{channel,alloc,0,1},1},
     {{channel,free,1,1},1},
     {{channel,init,1,1},1},
     {{channel,handle_call,3,1},1},
     {{channel,handle_call,3,2},1},
     {{channel,handle_call,3,3},1},
     {{channel,terminate,2,1},1},
     {{channel,channels,0,1},1},
     {{channel,alloc,1,1},1},
     {{channel,alloc,1,2},0},
     {{channel,free,2,1},1}]}

For channel, the result shows that all clauses have been called once, except the second clause of channel:alloc/1 which has not been called at all.

Finally, if the analysis is made on line level, the result is given as a list of tuples {Line,Calls}. Here Calls is the number of times each line has been executed:

11> cover:analyse(channel,calls,line).
{ok,[{{channel,9},1},
     {{channel,12},1},
     {{channel,17},1},
     {{channel,20},1},
     {{channel,25},1},
     {{channel,28},1},
     {{channel,31},1},
     {{channel,32},1},
     {{channel,35},1},
     {{channel,36},1},
     {{channel,39},1},
     {{channel,44},1},
     {{channel,47},1},
     {{channel,49},0},
     {{channel,52},1}]}

For channel, the result shows that all lines have been executed once, except line number 49 which has not been executed at all.

Analysis to File

A line level calls analysis of channel can be written to a file using cover:analysis_to_file/1:

12> cover:analyse_to_file(channel).
{ok,"channel.COVER.out"}

The function creates a copy of channel.erl where it for each executable line is specified how many times that line has been executed. The output file is called channel.COVER.out.

File generated from channel.erl by COVER 2001-05-21 at 11:16:38

****************************************************************************

        |  -module(channel).
        |  -behaviour(gen_server).
        |  
        |  -export([start_link/0,stop/0]).
        |  -export([alloc/0,free/1]). % client interface
        |  -export([init/1,handle_call/3,terminate/2]). % callback functions
        |  
        |  start_link() ->
     1..|      gen_server:start_link({local,channel},channel,[],[]).
        |  
        |  stop() ->
     1..|      gen_server:call(channel,stop).
        |  
        |  %%%-Client interface functions------------------------------------
        |  
        |  alloc() ->
     1..|      gen_server:call(channel,alloc).
        |  
        |  free(Channel) ->
     1..|      gen_server:call(channel,{free,Channel}).
        |  
        |  %%%-gen_server callback functions---------------------------------
        |  
        |  init(_Arg) ->
     1..|      {ok,channels()}.
        |  
        |  handle_call(stop,Client,Channels) ->
     1..|      {stop,normal,ok,Channels};
        |  
        |  handle_call(alloc,Client,Channels) ->
     1..|      {Ch,Channels2} = alloc(Channels),
     1..|      {reply,{ok,Ch},Channels2};
        |  
        |  handle_call({free,Channel},Client,Channels) ->
     1..|      Channels2 = free(Channel,Channels),
     1..|      {reply,ok,Channels2}.
        |  
        |  terminate(_Reason,Channels) ->
     1..|      ok.
        |  
        |  %%%-Internal functions--------------------------------------------
        |  
        |  channels() ->
     1..|      [ch1,ch2,ch3].
        |  
        |  alloc([Channel|Channels]) ->
     1..|      {Channel,Channels};
        |  alloc([]) ->
     0..|      false.
        |  
        |  free(Channel,Channels) ->
     1..|      [Channel|Channels].

Conclusion

By looking at the results from the analyses, it can be deducted that the test case does not cover the case when all channels are allocated and test.erl should be extended accordingly.
Incidentally, when the test case is corrected a bug in channel should indeed be discovered.

When the Cover analysis is ready, Cover is stopped and all Cover compiled modules are unloaded. The code for channel is now loaded as usual from a .beam file in the current path.

13> code:which(channel).
cover_compiled
14> cover:stop().
ok
15> code:which(channel).
"./channel.beam"

1.3 Miscellaneous

Performance

Execution of code in Cover compiled modules is slower and more memory consuming than for regularly compiled modules. As the Cover database contains information about each executable line in each Cover compiled module, performance decreases proportionally to the size and number of the Cover compiled modules.

To improve performance when analysing cover results it is possible to do multiple calls to analyse and analyse_to_file at once. You can also use the async_analyse_to_file convenience function.

Executable Lines

Cover uses the concept of executable lines, which is lines of code containing an executable expression such as a matching or a function call. A blank line or a line containing a comment, function head or pattern in a case- or receive statement is not executable.

In the example below, lines number 2,4,6,8 and 11 are executable lines:

1: is_loaded(Module,Compiled) ->
2:   case get_file(Module,Compiled) of
3:     {ok,File} ->
4:       case code:which(Module) of
5:         ?TAG ->
6:           {loaded,File};
7:         _ ->
8:           unloaded
9:       end;
10:    false ->
11:      false
12:  end.

Code Loading Mechanism

When a module is Cover compiled, it is also loaded using the normal code loading mechanism of Erlang. This means that if a Cover compiled module is re-loaded during a Cover session, for example using c(Module), it will no longer be Cover compiled.

Use cover:is_compiled/1 or code:which/1 to see if a module is Cover compiled (and still loaded) or not.

When Cover is stopped, all Cover compiled modules are unloaded.

1.4 Using the Web Based User Interface to Cover

Introduction

To ease the use of Cover there is a web based user interface to Cover called WebCover. WebCover is designed to be started and used via WebTool. It is possible to Cover compile Erlang modules and to generate printable Cover and Call analyses via the web based user interface.

Start the Web Based User Interface to Cover

To start WebCover you can either start WebTool, point a browser to the start page of WebTool and start WebCover from there, or you can use the start_webtool script to start Webtool, WebCover and a browser. See WebTool documentation for further information.

Currently WebCover is only compatible with Internet Explorer and Netscape Navigator 4.0 and higher.

Navigating WebCover

From the menu in the lefthand frame you can select the Nodes, Compile, Import or Result page.

From the Nodes page you can add remote nodes to participate in the coverage analysis. Coverage data from all involved nodes will then be merged during analysis.

From the Compile page you can Cover compile .erl or .beam files.

From the Import page you can import coverage data from a previous analysis. Imported data will then be merged with the current coverage data. Note that it is only possible to import files with the extension .coverdata.

From the Result page you can analyse, reset or export coverage data.

Please follow the instructions on each page.