[erlang-questions] Load testing in parallel

[David Mitchell]

Great tip Serge!

Like many others, I'm new to Erlang - I've read through the
"Programming Erlang" book, "Thinking in Erlang" and various PDFs at
the main Erlang site.  I've written a few trivial bits of code,
identified and sorted out a bunch of problems I've created, and now
I'm moving into using Erlang for larger projects.  However, I've never
come across info about now() and timer.diff() before; like Ahmed, I've
been using statistics(wall_clock) for profiling purposes.

Where is that type of info documented?  Is it only in the various
library APIs (and thus I'll have to work through each of these to get
a handle on this type of info), or is there some "best practices"-type
documentation I haven't yet stumbled on?  I don't mind putting in the
time to do my research, but I suspect I'm not yet across all the
"good" sources of info.

Thanks in advance

Dave M.

On 13/08/07, Serge Aleynikov <saleyn@REDACTED> wrote:
> You have several problems with this code:
>
> 1. Since you don't provide implementation of generate_lists/3, I assume
>     it returns a flat list.  In this case in the start function you call:
>
>        spawn(?MODULE, run_process,  [Num, Op, Head, Pid]),
>
>     if here Head is not a list, whereas run_process/4 expects a list
>     as the third argument, lists:map/2 function would
>     crashes the process silently.  You can fix it by changing that line
>     to:
>        spawn(?MODULE, run_process,  [Num, Op, [Head], Pid]),
>
>
> 2. Also you use the following call to obtain time:
>
>     {_, Wallclock_Time_Since_Last_Call} = statistics(wall_clock).
>
>     Wallclock_Time_Since_Last_Call is time in milliseconds, so unless
>     evaluated function takes more than a millisecond you'd get a 0.
>
>     Moreover, unfortunately Wallclock_Time_Since_Last_Call is a *globally
>     shared* counter, so any process that calls statistics would cause a
>     reset of this value.  So in a concurrent system where many processes
>     use this function you'll likely always get a zero.
>
>     use T1 = now(), ..., T2 = now(), ... timer:now_diff(T2, T1)
>     to measure time.
>
> Serge
>
>
>
> Ahmed Ali wrote:
> > Hi all,
> >
> > I've been trying to load test a function in my code. The way I'm doing it is
> > to generate lists of the data I want to process, start a process for each
> > list and calculate runtime for each process. The function that I implemented
> > will call a  WebService operation in a different host for each data.
> >
> > I have the code below for this test. what I do is basically run
> > load_test(10, 1000) to generate 10 lists, each with 1000 data in it.
> >
> > The problem is that WebService call is done successfully but I don't get any
> > output for the statistics. When I run the code sequentially (i.e. instead of
> > spawn the call to run_process, I call run_process directly) I get the output
> > with no issues. Is there something that I missed in the code below? I
> > appreciate your help.
> >
> > Best regards,
> >
> > Ahmed Al-Issaei
> >
> > run_process(Num, Fun, List, _Pid) ->
> >     statistics(wall_clock),
> >     io:format("load testing process~n"),
> >     lists:map(Fun, List),
> >     {_, Time2} = statistics(wall_clock),
> >     U2 = Time2 / 1000,
> >     io:format("Num (~s) done: total time = ~p~n",[Num, U2]).
> >
> > start(_N, _Op, [], _Pid) ->
> >     true;
> > start(Num, Op, Final, Pid) ->
> >     [Head | Rest] = Final,
> >     spawn(?MODULE, run_process,  [Num, Op, Head, Pid]),
> >     start(Num-1, Op, Rest, Pid).
> >
> > load_test(Threads, Size) ->
> >     List = generate_lists(Threads, Size, []),
> >     Op = fun(X) -> call_ws_create(X) end,
> >     start(Threads, Op, List, self()).
> >
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>