Author: Björn Gustavsson <bjorn(at)erlang(dot)org>
Status: Final/R15B Proposal is implemented in OTP release R15B
Type: Standards Track
Created: 01-Mar-2011
Erlang-Version: R15A
Post-History:

EEP 36: Line numbers in exceptions

Abstract

Extend each entry in the call stack backtrace (hereafter called stacktrace) returned from the erlang:get_stacktrace/0 BIF and from the catch operator with filenames and line number information.

Specification

Currently a stack trace returned from erlang:get_stacktrace/0 (and the catch operator) is a list of three-tuples, where each tuple looks like:

{Module,Function,Arity}

(In some cases, the third element may be a list of arguments instead of the function arity.)

We propose to change each tuple to:

{Module,Function,Arity,LocationInfo}

LocationInfo is a property list (a list of two-tuples) that contains filename and line number information. If there is line number information available, the list will look like:

[{file,FilenameString},{line,LineNumber}]

The list should be accessed using proplists:get_value/3 or lists:keyfind/3, not by direct matching, since a future release may add more items to the list or change the order.

The filename is usually the same as the module with the extension ".erl" added, but if function definitions have been placed in a header file, the filename will be the name of the header file. The filename will also be different if the Erlang source file has been generated by a code generator such as yecc.

The line number will never be zero; instead LocationInfo will be set to an empty list.

The list will be empty if there is no location information available. Here are some reasons that location information may be missing:

Implementation requirements

This EEP does not specify exactly how line number information should be implemented, but it does impose some requirements on the implementation:

Example

In the examples, we will use the following module:

-module(example).
-export([m/1]).
-include("header.hrl").

m(L) ->
    {ok,lists:map(fun f/1, L)}.  %Line 6

and the header file header.hrl:

f(X) ->
    abs(X) + 1.        %Line 2

Using R14B01 to call our example module, we get the following result:

1> example:m([-1,0,1,2]).
{ok,[2,1,2,3]}
2> example:m([-1,0,1,2,not_a_number]).
** exception error: bad argument
     in function  abs/1
        called as abs(not_a_number)
     in call from example:f/1
     in call from lists:map/2
     in call from lists:map/2
     in call from example:m/1
3> catch example:m([-1,0,1,2,not_a_number]).
{'EXIT',{badarg,[{erlang,abs,[not_a_number]},
                 {example,f,1},
                 {lists,map,2},
                 {lists,map,2},
                 {example,m,1},
                 {erl_eval,do_apply,5},
                 {erl_eval,expr,5},
                 {shell,exprs,7}]}}

In a system with line number information enabled, we get:

1> example:m([-1,0,1,2]).             
{ok,[2,1,2,3]}
2> example:m([-1,0,1,2,not_a_number]).
** exception error: bad argument
     in function  abs/1
        called as abs(not_a_number)
     in call from example:f/1 (header.hrl, line 2)
     in call from lists:map/2 (lists.erl, line 948)
     in call from lists:map/2 (lists.erl, line 948)
     in call from example:m/1 (example.erl, line 6)
3> catch example:m([-1,0,1,2,not_a_number]).
{'EXIT',{badarg,[{erlang,abs,[not_a_number],[]},
                 {example,f,1,[{file,"header.hrl"},{line,2}]},
                 {lists,map,2,[{file,"lists.erl"},{line,948}]},
                 {lists,map,2,[{file,"lists.erl"},{line,948}]},
                 {example,m,1,[{file,"example.erl"},{line,6}]},
                 {erl_eval,do_apply,5,[{file,"erl_eval.erl"},{line,482}]},
                 {erl_eval,expr,5,[{file,"erl_eval.erl"},{line,276}]},
                 {shell,exprs,7,[{file,"shell.erl"},{line,666}]}]}}

If we compile the example module using the BEAM compiler in R14B01, there will not be any line number information for that module:

1> example:m([-1,0,1,2,not_a_number]).
** exception error: bad argument
     in function  abs/1
        called as abs(not_a_number)
     in call from example:f/1 
     in call from lists:map/2 (lists.erl, line 948)
     in call from lists:map/2 (lists.erl, line 948)
     in call from example:m/1 
2> catch example:m([-1,0,1,2,not_a_number]).
{'EXIT',{badarg,[{erlang,abs,[not_a_number],[]},
                 {example,f,1,[]},
                 {lists,map,2,[{file,"lists.erl"},{line,948}]},
                 {lists,map,2,[{file,"lists.erl"},{line,948}]},
                 {example,m,1,[]},
                 {erl_eval,do_apply,5,[{file,"erl_eval.erl"},{line,482}]},
                 {erl_eval,expr,5,[{file,"erl_eval.erl"},{line,276}]},
                 {shell,exprs,7,[{file,"shell.erl"},{line,666}]}]}}

Motivation

The lack of line number information in exceptions is a major stumbling block for many beginners, and is a time waster for experienced Erlang programmers.

An often repeated piece of advice to mitigate the lack of line number information is to write smaller functions. To some extent, that is good advice, but some functions are most naturally written as a single function with many clauses. One example is the handle_call/3 callback for a gen_server process. Another example is test suites. In a typical test suite, every line tests a condition and can potentially fail. It is not practical to put every line that may fail in a separate function.

Test suites based on common_test are automatically run through a parse transform that provides line number information when an exception occurs. The parse transform inserts before every line code that saves the current function name and line number in the process dictionary. When an exception occurs, the line number can be retrieved and presented.

One problem with this approach is that the test suite will run slower, which can cause test cases to fail if timeouts expire in the system being tested. Another problem is that by default the parse transform is only run on the test modules themselves, and therefore exceptions that occur in other parts of the code (support libraries for testing or the product itself) does not have any line information.

Rationale

We have chosen to let erlang:get_stacktrace/0 and the catch operator return stacktraces with filename and line number information (instead of introducing a new function called, for example, erlang:get_full_stacktrace/3). That means that code that simply passes on the stacktrace (to erlang:raise/3) does not need to be updated. For example, the following code that catches an exception, logs it, and pass it on does not need to be updated:

try
    some_call_that_may_fail()
catch
    Class:Reason ->
        Stk = erlang:get_stacktrace(),
        log(Class, Reason, Stk),
        erlang:raise(Class, Reason, Stk)
end

One the other hand, that means that code that assumes that the stacktrace only may contain three-tuples will no longer work and needs to be updated.

There are several reasons for the requirement that the line number information should be loaded by default (rather than ordered by giving an option).

Therefore it is better that the developers that cannot afford any increase in the size of the loaded code are the ones that must give an option to turn off loading of line number information.

Backwards Compatibility

Applications that examine the stacktrace and assume that it contains three-tuples must be updated. The erlang:raise/3 BIF still accepts three-tuples (it will translate those to four tuples with an empty list in the fourth element); thus it is not mandatory to update calls to erlang:raise/3.

Implementation

The reference implementation can be fetched from Github like this:

git fetch git://github.com/bjorng/otp.git bjorn/line-numbers-in-exceptions

Here is an overview of the implementation:

The BEAM compiler inserts a line instruction before every construct that may generate an exception and before every call that will be included in the stacktrace. (Local tail-recursive calls need no line instruction, but external tail-recursive calls need a line instruction because they may be calls to BIFs.)

The line instruction has a single operand, an index into a line number table. The line number table is stored in the "Line" chunk in the BEAM file. The "Line" chunk and line instructions increase the file size of BEAM files by about five percent.

The loader will remove the line instructions from the code that will be executed, but will remember their location and create a table sorted in address order mapping from program counter to line number information. When a stacktrace needs to be built, the run-time system will do a binary search for the program counter of exception-causing instruction and each continuation pointer.

For the benefit of embedded system that run in a very constrained memory space, the run-time system can be started with the '+L' option to disable loading of the line number information. The code will still be about one percent larger than code compiled without line number information, because the compiler was unable to do code sharing optimizations on instructions that cause exceptions (such as the badmatch instruction).

In the current implementation, the line number information increases the size of the loaded code by roughly ten percent.

Copyright

This document has been placed in the public domain.