[erlang-questions] Parallel Shootout & a style question

[Gleb Peregud]

On Thu, Sep 4, 2008 at 1:21 PM, Mats Cronqvist <mats.cronqvist@REDACTED> wrote:
> Gleb Peregud wrote:
>>
>> It would be nice to include plists (or similar) into stdlib
>> permanently. It will allow users of Erlang/OTP fine-tune their
>> applications for SMP systems just by adding "p" (and Malt) where
>> necessary. I think this issue should be in the hands of the
>> programmer.
>
>  that the competent programmer will always be able to tune her application
> to however many cores she's targeting is a truism. the issue at hand is what
> strategy the OTP libraries should use when the programmer does not/cannot
> provide hints (or when running legacy code).
>
>> Automatic "parallelization" of lists:* will not be
>> efficient, at least without extensive compile-time (or even run-time)
>> analysis of the code (if this is possible/feasible at all).
>
>  "will not" is strong wording. can you provide some evidence to back that
> up?

You are right, I've used too strong words here. I'm not able to back them up.

These are just my thoughts/intuition on the subject :) The following
are the same. There's a lot of different situations, where for example
lists:map may be used.

First example: It may be used to run CPU intensive computations on few
(comparable to the number of the cores) elements - in such a situation
it would be feasible to parallelize it with one process per element.

Second example: But when we are talking about hundreds of thousands
(N) of elements in a list, where computations for each is not CPU
intensive, it makes sense to run P ( = k*C, where C is the number of
cores, k is a small integer) processes to process chunks of size N/P.
If done in other way (for example one process for each element) the
overhead of spawning processes would be too big.

My wording was based on such way of thinking. That's why I think there
should be no automatic parallelism there and the only person to decide
to parallelise is the programmer...

Sorry if I'm talking truisms, just ignore the message in this case :)

>
>  mats



-- 
Gleb Peregud
http://gleber.pl/

Every minute is to be grasped.
Time waits for nobody.
-- Inscription on a Zen Gong