[erlang-questions] Game of Erlang life [was: Load testing in parallel]

Serge Aleynikov saleyn@REDACTED
Wed Aug 15 13:30:55 CEST 2007

Of course your own "circles" can be different from what was described - 
each developer has his own degree of interest, pace and learning curve.

If you are approaching Erlang from "number crunching" crunching 
perspective from the very beginning it might be a disappointment to see 
10x performance drop compared to C/C++/Pascal family.  But in doing so, 
you may be missing the point - Erlang is a domain-specific language. 
Despite the fact that it's a general-purpose language - there are parts 
of it that really excel - namely its approach to concurrency, 
declarative style and robustness through OTP.  As Scott mentioned in one 
of the recent emails - you really begin appreciating this robustness 
when your programs work reliably and your pager is quiet.


Ahmed Ali wrote:
> Hi Serge,
> Actually, I'm not sure if the last 3 "circles" are seperate at all. It seems
> to me that "number crunching" circle could be in the first part when
> learning Erlang. Same for the 4th one. Are you referring to implementation
> of the last 2 circles when you divide it to different stages?
> Best regards,
> Ahmed Al-Issaei
> On 8/14/07, Serge Aleynikov <saleyn@REDACTED> wrote:
>> So far the best resource of OTP documentation is the on-line reference:
>> http://www.erlang.org/doc.
>> Sometimes this documentation may seem odd or vague for newcomers, yet
>> the more you learn Erlang/OTP the more you understand and appreciate the
>> format, depth and value of the content.
>> Once you mastered first "Programming Erlang" chapters (ch.1-15,
>> excluding OTP ones) I suggest that you start by studying on-line module
>> documentation under "Basic Applications": stdlib and kernel.  They
>> expose a wealth of functions, many of which you'll find used in all
>> projects.  Despite the fact that it may seem like a lot of information,
>> if you approach this studying methodically and patiently, you'll be able
>> to filter content on the module level first (i.e. gaining high-level
>> understanding of what each module does) and then in those modules that
>> seem to trigger more of your interest (like kernel/erlang, stdlib/io,
>> stdlib/ets, etc.) study documented functions.
>> Don't worry about OTP (aside from stdlib and kernel applications) as
>> much in the beginning - when you feel you have a good grasp on basic
>> functions available in the two apps above, read the Design Principles
>> guide [1] to understand OTP behaviors - which is the "meat" part of the
>> framework that gives you the power of increasing your coding
>> productivity by implying practical and reliable abstractions.
>> At this point, go back to "Programming Erlang" and study the OTP-related
>> chapters (ch.16 onward) that give you application examples of how to use
>>   behaviors.  This will conclude your "first circle of Erlang life" and
>> give you enough knowledge to move on to exploiting the power of
>> Erlang/OTP in your projects.
>> The "second circle of Erlang life" will begin when your curiosity will
>> drive you to understanding that Erlang is *open source* and you'll start
>> exploring the sources of OTP applications included in distribution.
>> That is when you'll actually begin seeing that on-line documentation is
>> rather good.  When you are comfortable with OTP code you may find bugs
>> and post them to the erlang-bugs@REDACTED list, and also make some
>> contributions.
>> The "third circle of Erlang life" will begin when you discover that
>> Erlang's performance sucks at number-crunching applications (which are
>> not the domain of problems Erlang was designed for), and begin writing
>> ports and drivers to overcome these bottlenecks.  Additionally As a
>> conclusion of this circle you'll be quite fascinated at how much thought
>> was put in preventing mis-behaving user code crash the Erlang emulator.
>> The "forth circle of Erlang life" will begin when your curiosity drives
>> you to studying sources of the Erlang emulator.  Here you'll learn how
>> I/O multiplexing, garbage collection, bytecode execution, etc. is done,
>> and answer some of more advanced questions you might have accumulated
>> about features of the run-time system.
>> By this time others will find that you are "... more object oriented in
>> thinking than most of his C++ guys" [2]. The "next circle of Erlang
>> life" will begin when you manage to get a job that will let your
>> evolution as an Erlang programmer use acquired knowledge during
>> day-time.  Since not many companies know the hidden powers of Erlang,
>> this will largely depend on your abilities to show the strength of
>> Erlang/OTP development to your manager, find suitable projects, and
>> generate spin to share this knowledge and apply it in your organization.
>> Regards,
>> Serge
>> [1] http://www.erlang.org/doc/design_principles/part_frame.html
>> [2]
>> http://www.erlang.org/pipermail/erlang-questions/2007-August/028392.html
>> David Mitchell wrote:
>>> Great tip Serge!
>>> Like many others, I'm new to Erlang - I've read through the
>>> "Programming Erlang" book, "Thinking in Erlang" and various PDFs at
>>> the main Erlang site.  I've written a few trivial bits of code,
>>> identified and sorted out a bunch of problems I've created, and now
>>> I'm moving into using Erlang for larger projects.  However, I've never
>>> come across info about now() and timer.diff() before; like Ahmed, I've
>>> been using statistics(wall_clock) for profiling purposes.
>>> Where is that type of info documented?  Is it only in the various
>>> library APIs (and thus I'll have to work through each of these to get
>>> a handle on this type of info), or is there some "best practices"-type
>>> documentation I haven't yet stumbled on?  I don't mind putting in the
>>> time to do my research, but I suspect I'm not yet across all the
>>> "good" sources of info.
>>> Thanks in advance
>>> Dave M.
>>> On 13/08/07, Serge Aleynikov <saleyn@REDACTED> wrote:
>>>> You have several problems with this code:
>>>> 1. Since you don't provide implementation of generate_lists/3, I assume
>>>>     it returns a flat list.  In this case in the start function you
>> call:
>>>>        spawn(?MODULE, run_process,  [Num, Op, Head, Pid]),
>>>>     if here Head is not a list, whereas run_process/4 expects a list
>>>>     as the third argument, lists:map/2 function would
>>>>     crashes the process silently.  You can fix it by changing that line
>>>>     to:
>>>>        spawn(?MODULE, run_process,  [Num, Op, [Head], Pid]),
>>>> 2. Also you use the following call to obtain time:
>>>>     {_, Wallclock_Time_Since_Last_Call} = statistics(wall_clock).
>>>>     Wallclock_Time_Since_Last_Call is time in milliseconds, so unless
>>>>     evaluated function takes more than a millisecond you'd get a 0.
>>>>     Moreover, unfortunately Wallclock_Time_Since_Last_Call is a
>> *globally
>>>>     shared* counter, so any process that calls statistics would cause a
>>>>     reset of this value.  So in a concurrent system where many
>> processes
>>>>     use this function you'll likely always get a zero.
>>>>     use T1 = now(), ..., T2 = now(), ... timer:now_diff(T2, T1)
>>>>     to measure time.
>>>> Serge
>>>> Ahmed Ali wrote:
>>>>> Hi all,
>>>>> I've been trying to load test a function in my code. The way I'm doing
>> it is
>>>>> to generate lists of the data I want to process, start a process for
>> each
>>>>> list and calculate runtime for each process. The function that I
>> implemented
>>>>> will call a  WebService operation in a different host for each data.
>>>>> I have the code below for this test. what I do is basically run
>>>>> load_test(10, 1000) to generate 10 lists, each with 1000 data in it.
>>>>> The problem is that WebService call is done successfully but I don't
>> get any
>>>>> output for the statistics. When I run the code sequentially (i.e.
>> instead of
>>>>> spawn the call to run_process, I call run_process directly) I get the
>> output
>>>>> with no issues. Is there something that I missed in the code below? I
>>>>> appreciate your help.
>>>>> Best regards,
>>>>> Ahmed Al-Issaei
>>>>> run_process(Num, Fun, List, _Pid) ->
>>>>>     statistics(wall_clock),
>>>>>     io:format("load testing process~n"),
>>>>>     lists:map(Fun, List),
>>>>>     {_, Time2} = statistics(wall_clock),
>>>>>     U2 = Time2 / 1000,
>>>>>     io:format("Num (~s) done: total time = ~p~n",[Num, U2]).
>>>>> start(_N, _Op, [], _Pid) ->
>>>>>     true;
>>>>> start(Num, Op, Final, Pid) ->
>>>>>     [Head | Rest] = Final,
>>>>>     spawn(?MODULE, run_process,  [Num, Op, Head, Pid]),
>>>>>     start(Num-1, Op, Rest, Pid).
>>>>> load_test(Threads, Size) ->
>>>>>     List = generate_lists(Threads, Size, []),
>>>>>     Op = fun(X) -> call_ws_create(X) end,
>>>>>     start(Threads, Op, List, self()).
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions

More information about the erlang-questions mailing list