[erlang-questions] Profiling tool to find bottleneck on my Erlang code

[Geoff Cant]

On 2011-10-06, at 11:30 , Zabrane Mickael wrote:

> Hi guys,
> 
> We have at work, a heavily parallel Erlang application running for years now.
> We would like to concentrate our efforts in the next couple of months in optimizing it bit more (if possible).
> 
> Could someone please point me to a good "high level profiling tool" (not eprof/fprof which we already now,
> and find not very sexy) to tracks bottlenecks?
> 
> Any advice will be appreciated.
> 
> Regards,
> Zabrane


It sounds like your system runs without crashing, which I'm guessing means you don't have a memory bottleneck. This means you probably have something that's CPU bound, or if you're not CPU bound, you probably have an inefficient messaging pattern somewhere that is hampering you.

To figure out if you have a CPU problem, I'd take a look at etop/dtop/entop and see if there are processes consuming a lot of reductions. If they're consuming these reductions in proportion to receiving a lot of input messages then they may be fine, but you can look at having multiple processes handle these messages in parallel. If they're just consuming lots of reductions, then check to see if the code in those processes can be made more efficient.

Finding a messaging pattern problem is harder - typically you'll need to have an idea of which processes might be involved, and then you need to trace them with dbg or something. Then expect to spend a bunch of time looking over the trace log to figure out how many messages it takes between which processes in order to complete an operation. Then you can figure out if there's a way to do it better.


So tool wise - use etop/dtop/entop first to figure out if you have CPU-bound hotspots, then use dbg tracing to investigate further.

Cheers,
--
Geoff Cant