[erlang-questions] Load testing in parallel

[Ahmed Ali]

Hi All,

Thanks for the info. It was really useful for me.

Serge,

Your assumption is true about  the list, it is flat list. Also, there's no
issue what so ever with spawn call, because it is working correctly. It's
just the time that wouldn't output. timer:now_diff/2 is very useful and it
is exactly what I've been looking for.

I discovered that i should change the last line in run_process/4 to:
io:format("Num ~p done: total time = ~p~n",[Num, U2]).
(i.e. ~p instead of ~s)          :)

Best regards,

Ahmed Al-Issaei

On 8/14/07, David Mitchell <monch1962@REDACTED> wrote:
>
> Great tip Serge!
>
> Like many others, I'm new to Erlang - I've read through the
> "Programming Erlang" book, "Thinking in Erlang" and various PDFs at
> the main Erlang site.  I've written a few trivial bits of code,
> identified and sorted out a bunch of problems I've created, and now
> I'm moving into using Erlang for larger projects.  However, I've never
> come across info about now() and timer.diff() before; like Ahmed, I've
> been using statistics(wall_clock) for profiling purposes.
>
> Where is that type of info documented?  Is it only in the various
> library APIs (and thus I'll have to work through each of these to get
> a handle on this type of info), or is there some "best practices"-type
> documentation I haven't yet stumbled on?  I don't mind putting in the
> time to do my research, but I suspect I'm not yet across all the
> "good" sources of info.
>
> Thanks in advance
>
> Dave M.
>
> On 13/08/07, Serge Aleynikov <saleyn@REDACTED> wrote:
> > You have several problems with this code:
> >
> > 1. Since you don't provide implementation of generate_lists/3, I assume
> >     it returns a flat list.  In this case in the start function you
> call:
> >
> >        spawn(?MODULE, run_process,  [Num, Op, Head, Pid]),
> >
> >     if here Head is not a list, whereas run_process/4 expects a list
> >     as the third argument, lists:map/2 function would
> >     crashes the process silently.  You can fix it by changing that line
> >     to:
> >        spawn(?MODULE, run_process,  [Num, Op, [Head], Pid]),
> >
> >
> > 2. Also you use the following call to obtain time:
> >
> >     {_, Wallclock_Time_Since_Last_Call} = statistics(wall_clock).
> >
> >     Wallclock_Time_Since_Last_Call is time in milliseconds, so unless
> >     evaluated function takes more than a millisecond you'd get a 0.
> >
> >     Moreover, unfortunately Wallclock_Time_Since_Last_Call is a
> *globally
> >     shared* counter, so any process that calls statistics would cause a
> >     reset of this value.  So in a concurrent system where many processes
> >     use this function you'll likely always get a zero.
> >
> >     use T1 = now(), ..., T2 = now(), ... timer:now_diff(T2, T1)
> >     to measure time.
> >
> > Serge
> >
> >
> >
> > Ahmed Ali wrote:
> > > Hi all,
> > >
> > > I've been trying to load test a function in my code. The way I'm doing
> it is
> > > to generate lists of the data I want to process, start a process for
> each
> > > list and calculate runtime for each process. The function that I
> implemented
> > > will call a  WebService operation in a different host for each data.
> > >
> > > I have the code below for this test. what I do is basically run
> > > load_test(10, 1000) to generate 10 lists, each with 1000 data in it.
> > >
> > > The problem is that WebService call is done successfully but I don't
> get any
> > > output for the statistics. When I run the code sequentially (i.e.
> instead of
> > > spawn the call to run_process, I call run_process directly) I get the
> output
> > > with no issues. Is there something that I missed in the code below? I
> > > appreciate your help.
> > >
> > > Best regards,
> > >
> > > Ahmed Al-Issaei
> > >
> > > run_process(Num, Fun, List, _Pid) ->
> > >     statistics(wall_clock),
> > >     io:format("load testing process~n"),
> > >     lists:map(Fun, List),
> > >     {_, Time2} = statistics(wall_clock),
> > >     U2 = Time2 / 1000,
> > >     io:format("Num (~s) done: total time = ~p~n",[Num, U2]).
> > >
> > > start(_N, _Op, [], _Pid) ->
> > >     true;
> > > start(Num, Op, Final, Pid) ->
> > >     [Head | Rest] = Final,
> > >     spawn(?MODULE, run_process,  [Num, Op, Head, Pid]),
> > >     start(Num-1, Op, Rest, Pid).
> > >
> > > load_test(Threads, Size) ->
> > >     List = generate_lists(Threads, Size, []),
> > >     Op = fun(X) -> call_ws_create(X) end,
> > >     start(Threads, Op, List, self()).
> > >
> > >
> > >
> > >
> ------------------------------------------------------------------------
> > >
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070814/4cb13bff/attachment.htm>