[erlang-questions] Erlang tracing

Jesper Louis Andersen <>
Tue Sep 22 12:47:38 CEST 2015

On Mon, Sep 21, 2015 at 10:52 AM, Lukas Larsson <> wrote:

> * Allow multiple tracers. Today only one port/process can be the receiver
> of trace data.

This would be especially welcome. You can run profiling next to
debug-tracing then, which is a nice thing.

>   * Better integration of dtrace/lttngt/systemtap into the erlang trace.

My advice would be: Study the DTrace model. Steal every good idea. In
DTrace, you enable probes, then define a filter set on those probes and
then you define what action (i.e., program) to run if the filter is passed.

Example: People have requested a "who calls me" query. Suppose we had a
probe in DTrace for when a function calls another function, in module 'm'

/ copyinstr(arg1) = "m" /
    caller = current_function()

    @callers[caller] = count()

This would in principle be enough to satisfy that query, if only we
extended DTrace probes a little bit. It would output a list of pairs
[{Caller, Count}].

Second observation: DTrace has something called a ustack(), which is the
backtrace stack in Userland. We can get that for the C layer in BEAM, but
we can't look into it for the Erlang world. The ustack() helpers are
D-scripts (they run in the kernel, and can only have finite loops among
other things). So we would probably need an easy way, in a struct, where we
can grab information about the frames on the emulator stack from the
kernel. Any tracing facility outside the Erlang world needs this.

Third observation: Simon's request is also easily satisfiable. Set a tick
timer at 997Hz from the kernel, then look up where we are once the timer
ticks. Preferably once per scheduler thread.

Key takeaway: Provide tools, not solutions. Once you can enable probes and
filter probes, a process can tie actions to probes like the above. We can
always push specialized optimizations down into the tracing layer for
efficiency, but we should go for a variant where we provide enabling tools
for programmers, rather than baked solutions.

I envision that things such as e2_debug, recon_trace, redbug and so on are
adaptable to a new model, easily.

Finally, we should really write a good dbg tutorial together with better
documentation. I'm willing to throw some hours into this if needed :)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150922/fd267630/attachment.html>

More information about the erlang-questions mailing list