[erlang-questions] Game of Erlang life [was: Load testing in parallel]
Serge Aleynikov
saleyn@REDACTED
Wed Aug 15 13:30:55 CEST 2007
Of course your own "circles" can be different from what was described -
each developer has his own degree of interest, pace and learning curve.
If you are approaching Erlang from "number crunching" crunching
perspective from the very beginning it might be a disappointment to see
10x performance drop compared to C/C++/Pascal family. But in doing so,
you may be missing the point - Erlang is a domain-specific language.
Despite the fact that it's a general-purpose language - there are parts
of it that really excel - namely its approach to concurrency,
declarative style and robustness through OTP. As Scott mentioned in one
of the recent emails - you really begin appreciating this robustness
when your programs work reliably and your pager is quiet.
Serge
Ahmed Ali wrote:
> Hi Serge,
>
> Actually, I'm not sure if the last 3 "circles" are seperate at all. It seems
> to me that "number crunching" circle could be in the first part when
> learning Erlang. Same for the 4th one. Are you referring to implementation
> of the last 2 circles when you divide it to different stages?
>
> Best regards,
>
> Ahmed Al-Issaei
>
> On 8/14/07, Serge Aleynikov <saleyn@REDACTED> wrote:
>> So far the best resource of OTP documentation is the on-line reference:
>> http://www.erlang.org/doc.
>>
>> Sometimes this documentation may seem odd or vague for newcomers, yet
>> the more you learn Erlang/OTP the more you understand and appreciate the
>> format, depth and value of the content.
>>
>> Once you mastered first "Programming Erlang" chapters (ch.1-15,
>> excluding OTP ones) I suggest that you start by studying on-line module
>> documentation under "Basic Applications": stdlib and kernel. They
>> expose a wealth of functions, many of which you'll find used in all
>> projects. Despite the fact that it may seem like a lot of information,
>> if you approach this studying methodically and patiently, you'll be able
>> to filter content on the module level first (i.e. gaining high-level
>> understanding of what each module does) and then in those modules that
>> seem to trigger more of your interest (like kernel/erlang, stdlib/io,
>> stdlib/ets, etc.) study documented functions.
>>
>> Don't worry about OTP (aside from stdlib and kernel applications) as
>> much in the beginning - when you feel you have a good grasp on basic
>> functions available in the two apps above, read the Design Principles
>> guide [1] to understand OTP behaviors - which is the "meat" part of the
>> framework that gives you the power of increasing your coding
>> productivity by implying practical and reliable abstractions.
>>
>> At this point, go back to "Programming Erlang" and study the OTP-related
>> chapters (ch.16 onward) that give you application examples of how to use
>> behaviors. This will conclude your "first circle of Erlang life" and
>> give you enough knowledge to move on to exploiting the power of
>> Erlang/OTP in your projects.
>>
>> The "second circle of Erlang life" will begin when your curiosity will
>> drive you to understanding that Erlang is *open source* and you'll start
>> exploring the sources of OTP applications included in distribution.
>> That is when you'll actually begin seeing that on-line documentation is
>> rather good. When you are comfortable with OTP code you may find bugs
>> and post them to the erlang-bugs@REDACTED list, and also make some
>> contributions.
>>
>> The "third circle of Erlang life" will begin when you discover that
>> Erlang's performance sucks at number-crunching applications (which are
>> not the domain of problems Erlang was designed for), and begin writing
>> ports and drivers to overcome these bottlenecks. Additionally As a
>> conclusion of this circle you'll be quite fascinated at how much thought
>> was put in preventing mis-behaving user code crash the Erlang emulator.
>>
>> The "forth circle of Erlang life" will begin when your curiosity drives
>> you to studying sources of the Erlang emulator. Here you'll learn how
>> I/O multiplexing, garbage collection, bytecode execution, etc. is done,
>> and answer some of more advanced questions you might have accumulated
>> about features of the run-time system.
>>
>> By this time others will find that you are "... more object oriented in
>> thinking than most of his C++ guys" [2]. The "next circle of Erlang
>> life" will begin when you manage to get a job that will let your
>> evolution as an Erlang programmer use acquired knowledge during
>> day-time. Since not many companies know the hidden powers of Erlang,
>> this will largely depend on your abilities to show the strength of
>> Erlang/OTP development to your manager, find suitable projects, and
>> generate spin to share this knowledge and apply it in your organization.
>>
>> Regards,
>>
>> Serge
>>
>>
>> [1] http://www.erlang.org/doc/design_principles/part_frame.html
>> [2]
>> http://www.erlang.org/pipermail/erlang-questions/2007-August/028392.html
>>
>>
>> David Mitchell wrote:
>>> Great tip Serge!
>>>
>>> Like many others, I'm new to Erlang - I've read through the
>>> "Programming Erlang" book, "Thinking in Erlang" and various PDFs at
>>> the main Erlang site. I've written a few trivial bits of code,
>>> identified and sorted out a bunch of problems I've created, and now
>>> I'm moving into using Erlang for larger projects. However, I've never
>>> come across info about now() and timer.diff() before; like Ahmed, I've
>>> been using statistics(wall_clock) for profiling purposes.
>>>
>>> Where is that type of info documented? Is it only in the various
>>> library APIs (and thus I'll have to work through each of these to get
>>> a handle on this type of info), or is there some "best practices"-type
>>> documentation I haven't yet stumbled on? I don't mind putting in the
>>> time to do my research, but I suspect I'm not yet across all the
>>> "good" sources of info.
>>>
>>> Thanks in advance
>>>
>>> Dave M.
>>>
>>> On 13/08/07, Serge Aleynikov <saleyn@REDACTED> wrote:
>>>> You have several problems with this code:
>>>>
>>>> 1. Since you don't provide implementation of generate_lists/3, I assume
>>>> it returns a flat list. In this case in the start function you
>> call:
>>>> spawn(?MODULE, run_process, [Num, Op, Head, Pid]),
>>>>
>>>> if here Head is not a list, whereas run_process/4 expects a list
>>>> as the third argument, lists:map/2 function would
>>>> crashes the process silently. You can fix it by changing that line
>>>> to:
>>>> spawn(?MODULE, run_process, [Num, Op, [Head], Pid]),
>>>>
>>>>
>>>> 2. Also you use the following call to obtain time:
>>>>
>>>> {_, Wallclock_Time_Since_Last_Call} = statistics(wall_clock).
>>>>
>>>> Wallclock_Time_Since_Last_Call is time in milliseconds, so unless
>>>> evaluated function takes more than a millisecond you'd get a 0.
>>>>
>>>> Moreover, unfortunately Wallclock_Time_Since_Last_Call is a
>> *globally
>>>> shared* counter, so any process that calls statistics would cause a
>>>> reset of this value. So in a concurrent system where many
>> processes
>>>> use this function you'll likely always get a zero.
>>>>
>>>> use T1 = now(), ..., T2 = now(), ... timer:now_diff(T2, T1)
>>>> to measure time.
>>>>
>>>> Serge
>>>>
>>>>
>>>>
>>>> Ahmed Ali wrote:
>>>>> Hi all,
>>>>>
>>>>> I've been trying to load test a function in my code. The way I'm doing
>> it is
>>>>> to generate lists of the data I want to process, start a process for
>> each
>>>>> list and calculate runtime for each process. The function that I
>> implemented
>>>>> will call a WebService operation in a different host for each data.
>>>>>
>>>>> I have the code below for this test. what I do is basically run
>>>>> load_test(10, 1000) to generate 10 lists, each with 1000 data in it.
>>>>>
>>>>> The problem is that WebService call is done successfully but I don't
>> get any
>>>>> output for the statistics. When I run the code sequentially (i.e.
>> instead of
>>>>> spawn the call to run_process, I call run_process directly) I get the
>> output
>>>>> with no issues. Is there something that I missed in the code below? I
>>>>> appreciate your help.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Ahmed Al-Issaei
>>>>>
>>>>> run_process(Num, Fun, List, _Pid) ->
>>>>> statistics(wall_clock),
>>>>> io:format("load testing process~n"),
>>>>> lists:map(Fun, List),
>>>>> {_, Time2} = statistics(wall_clock),
>>>>> U2 = Time2 / 1000,
>>>>> io:format("Num (~s) done: total time = ~p~n",[Num, U2]).
>>>>>
>>>>> start(_N, _Op, [], _Pid) ->
>>>>> true;
>>>>> start(Num, Op, Final, Pid) ->
>>>>> [Head | Rest] = Final,
>>>>> spawn(?MODULE, run_process, [Num, Op, Head, Pid]),
>>>>> start(Num-1, Op, Rest, Pid).
>>>>>
>>>>> load_test(Threads, Size) ->
>>>>> List = generate_lists(Threads, Size, []),
>>>>> Op = fun(X) -> call_ws_create(X) end,
>>>>> start(Threads, Op, List, self()).
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
>
More information about the erlang-questions
mailing list