[erlang-questions] process stats

[Siraaj Khandkar]

On Nov 2, 2012, at 12:39 AM, József Bérces wrote:

> Hi,
> 
> I am writing an application that runs in the background crunching some data and writing result files to the disk. It has a web interface to configure what data and how to process. And I was thinking about that it would be good to be able to present some stats on the web interface about which Erlang processes how much CPU resources used during the last second/minute/day/week etc.
> 
> Is there any way to get this kind of info somehow without wasting too much resources on the stat collection itself? I do not want it for all Erlang processes, only for a well defined subset of them.
> 
> eprof/fprof seem to be too heavy: they would take too much system resources and they would produce too detailed data for me.
> 
> erlang:system_profile seems to be promising (I have not started playing with it) but it has that note in the doc that it is experimental so I am a bit afraid of it.
> 
> I was also thinking about tracing my processes with the flag 'running' so I would get 'in' and 'out' message tags in a profiling process. I am not sure how much resources this would take.
> 
> Any suggestions are welcome!


Generally, a VM process is not mapped to an OS process, so there's no way to
get an actual, physical CPU utilization of an individual virtual process.

I imagine there should be a way to lookup proportional scheduler time of a
virtual process, but, like you, I'm not seeing anything obvious on the erlang
man page either :) Hopefully someone more knowledgeable will chime in.

That said, it may be worth considering outsourcing CPU-intensive data crunching
to an external process. Besides the performance, it also eases monitoring.

I'm working with a similar scenario (coordination in Erlang, but outsourcing
crunching to OCaml) and using tools from the sysstat package on GNU/Linux to
monitor the external process.

For example:

    Job = "/path/to/data_to_crunch"
    PortSettings = [exit_status, stderr_to_stdout],

    WorkerCmd = "/myproject/bin/process_data " ++ Job,
    WorkerPort = open_port({spawn, WorkerCmd}, PortSettings),
    {os_pid, WorkerOSPID} = erlang:port_info(WorkerPort, os_pid),

    MonitorInterval = "1",
    MonitorCmd = string:join(["pidstat", "-u", "-p", integer_to_list(WorkerOSPID), Interval], " "),
    MonitorPort = open_port({spawn, MonitorCmd}, PortSettings).

You can then parse messages like this:

    receive
        {#Port<0.xx>, {data, "12:50:40 PM      1234    99.00    0.00    99.00     3  do_heavy_stuff\n"}}
    end


-- 
Siraaj Khandkar
.o.
..o
ooo