[erlang-questions] Load testing in parallel
Ulf Wiger (TN/EAB)
ulf.wiger@REDACTED
Tue Aug 14 11:57:27 CEST 2007
I think there could be a chapter in the efficiency
guide about simple benchmarking. There is a chapter,
but it feels a bit complicated.
Personally, I am fond of timer:tc(M, F, A). It's a very
convenient way to get a rough picture of the cost of
different operations.
It's not very useful for very cheap functions. Perhaps
one could add a timer:tc(N, M, F, A), which calls
apply(M, F, A) N times, and reports how long it took?
-module(timer1).
-export([tc/4]).
tc(N, M, F, A) when N > 0 ->
Pid = spawn_opt(fun() ->
exit(call_acc(N, M, F, A, erlang:now(), []))
end, [{min_heap_size, 10000}]),
MRef = erlang:monitor(process, Pid),
receive
{'DOWN', MRef, process, _, Result} ->
Result
end.
call_acc(N, M, F, A, St, Acc) when N > 0 ->
call_acc(N-1, M, F, A, St, [catch apply(M, F, A)|Acc]);
call_acc(0, _, _, _, Start, Acc) ->
Stop = erlang:now(),
{timer:now_diff(Stop, Start), lists:reverse(Acc)}.
Example:
2> timer:tc(lists,reverse,[[1,2,3,4,5,6,7,8,9,10]]).
{6,[10,9,8,7,6,5,4,3,2,1]}
3> timer1:tc(10, lists,reverse, [lists:seq(1,10)]).
{12,
[[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1]]}
4> timer1:tc(100, lists,reverse, [lists:seq(1,10)]).
{90,
[[10,9,8,7,6,5,4,3,2,1], ...
5> timer1:tc(1000, lists,reverse, [lists:seq(1,10)]).
{2121,
[[10,9,8,7,6,5,4,3,2,1],
[10,9,8,7,6,5,4,3,2,1], ...
6> {_, Term} = timer1:tc(1000, lists,reverse,...).
...
7> erts_debug:flat_size(Term)
22000
Showing that the result is quite different depending
on how many iterations we measure on:
#iter time/iter
1 6
10 3
100 0.9
1000 2.1
The accumulated value over 1000 iterations is larger
than the heap size of the process, which means that
GC cost is included in the last measurement.
The included module turns off accumulation by default,
and adds a tc/5, with the option whether or not to
accumulate. In the case of no accumulation, the last
result is returned.
It's not perfect by any means, but I find it useful
for quick measurements in the erlang shell.
BR,
Ulf W
> -----Original Message-----
> From: erlang-questions-bounces@REDACTED
> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of
> David Mitchell
> Sent: den 14 augusti 2007 09:58
> To: Erlang
> Subject: Re: [erlang-questions] Load testing in parallel
>
> Great tip Serge!
>
> Like many others, I'm new to Erlang - I've read through the
> "Programming Erlang" book, "Thinking in Erlang" and various
> PDFs at the main Erlang site. I've written a few trivial
> bits of code, identified and sorted out a bunch of problems
> I've created, and now I'm moving into using Erlang for larger
> projects. However, I've never come across info about now()
> and timer.diff() before; like Ahmed, I've been using
> statistics(wall_clock) for profiling purposes.
>
> Where is that type of info documented? Is it only in the
> various library APIs (and thus I'll have to work through each
> of these to get a handle on this type of info), or is there
> some "best practices"-type documentation I haven't yet
> stumbled on? I don't mind putting in the time to do my
> research, but I suspect I'm not yet across all the "good"
> sources of info.
>
> Thanks in advance
>
> Dave M.
>
> On 13/08/07, Serge Aleynikov <saleyn@REDACTED> wrote:
> > You have several problems with this code:
> >
> > 1. Since you don't provide implementation of
> generate_lists/3, I assume
> > it returns a flat list. In this case in the start
> function you call:
> >
> > spawn(?MODULE, run_process, [Num, Op, Head, Pid]),
> >
> > if here Head is not a list, whereas run_process/4 expects a list
> > as the third argument, lists:map/2 function would
> > crashes the process silently. You can fix it by
> changing that line
> > to:
> > spawn(?MODULE, run_process, [Num, Op, [Head], Pid]),
> >
> >
> > 2. Also you use the following call to obtain time:
> >
> > {_, Wallclock_Time_Since_Last_Call} = statistics(wall_clock).
> >
> > Wallclock_Time_Since_Last_Call is time in milliseconds,
> so unless
> > evaluated function takes more than a millisecond you'd get a 0.
> >
> > Moreover, unfortunately Wallclock_Time_Since_Last_Call
> is a *globally
> > shared* counter, so any process that calls statistics
> would cause a
> > reset of this value. So in a concurrent system where
> many processes
> > use this function you'll likely always get a zero.
> >
> > use T1 = now(), ..., T2 = now(), ... timer:now_diff(T2, T1)
> > to measure time.
> >
> > Serge
> >
> >
> >
> > Ahmed Ali wrote:
> > > Hi all,
> > >
> > > I've been trying to load test a function in my code. The way I'm
> > > doing it is to generate lists of the data I want to
> process, start a
> > > process for each list and calculate runtime for each process. The
> > > function that I implemented will call a WebService
> operation in a different host for each data.
> > >
> > > I have the code below for this test. what I do is basically run
> > > load_test(10, 1000) to generate 10 lists, each with 1000
> data in it.
> > >
> > > The problem is that WebService call is done successfully
> but I don't
> > > get any output for the statistics. When I run the code
> sequentially
> > > (i.e. instead of spawn the call to run_process, I call
> run_process
> > > directly) I get the output with no issues. Is there
> something that I
> > > missed in the code below? I appreciate your help.
> > >
> > > Best regards,
> > >
> > > Ahmed Al-Issaei
> > >
> > > run_process(Num, Fun, List, _Pid) ->
> > > statistics(wall_clock),
> > > io:format("load testing process~n"),
> > > lists:map(Fun, List),
> > > {_, Time2} = statistics(wall_clock),
> > > U2 = Time2 / 1000,
> > > io:format("Num (~s) done: total time = ~p~n",[Num, U2]).
> > >
> > > start(_N, _Op, [], _Pid) ->
> > > true;
> > > start(Num, Op, Final, Pid) ->
> > > [Head | Rest] = Final,
> > > spawn(?MODULE, run_process, [Num, Op, Head, Pid]),
> > > start(Num-1, Op, Rest, Pid).
> > >
> > > load_test(Threads, Size) ->
> > > List = generate_lists(Threads, Size, []),
> > > Op = fun(X) -> call_ws_create(X) end,
> > > start(Threads, Op, List, self()).
> > >
> > >
> > >
> > >
> --------------------------------------------------------------------
> > > ----
> > >
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: timer1.erl
Type: application/octet-stream
Size: 936 bytes
Desc: timer1.erl
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070814/0d8bf600/attachment.obj>
More information about the erlang-questions
mailing list