[erlang-questions] Representation of a map in bytecode?

[Kenneth Lundin]

Hi,

On 9/15/07, Hugh Perkins <hughperkins@REDACTED> wrote:
> How is a map represented in bytecode?  To what extent is it clearly
> identifiable as a map, directly from the bytecode?

There is nothing special that distinguishes a map in the byte code.
A map is typically implemented with a call to lists:map(Fun,List) or with
a list comprehension like this [mapping(X)|| X <- List].
Although it would be possible to find the pattern for a map in the byte code
I think it is a bad idea trying to parallellize this by some automatic magic.
One of the strengths with Erlang is that it is very easy and natural
to parallellize things explicitly by the use of Erlang processes. If a
system already is parallellized
into e.g. one Erlang process per connection in a web-server, one
Erlang process per ongoing call in a telecom system etc. there is very
little reason trying to parallellize each map if there are thousands
of call or connection processes
executing simultaneously and that each of those potentially will
perform a map operation as part of their work.

What I probably try to say is that there are different approaches to
parallellism.

1) Trying to optimize a specific algorithm by use of parallell
computation. Assuming that this is the only ongoing thing on this
computer.
This can be a heavy map operation for example, but even then it must
be considered if the actuall mapping work for each element is big
enough to motivate a split into many parallell tasks.

2) A SW system that is explicitly designed to be parallell from the
beginning (like the one process per connection or one process per call
above).
If the system design already allows thousands of parallell tasks it is
far from obvious that also the internal execution of each such task
should benefit from
automatic parallellisation. I actually believe that a system can be
over parallellized and that this will degrade the performance.

I don't think automatic parallellization of a map operation is
interesting when it is so easy to write and encapsulate that parallell
map into a function and call that function when parallellization is
wanted. This function can of course be included in
one of the standard libraries if the problem is common enough.

/Kenneth (Erlang/OTP team at Ericsson)
>
> Kindof contemplating to what extent it could be interesting/useful/fun
> to make a fork of Erlang that automatically parallelizes maps across
> available processor cores.
>
> This should probably be done dynamically, at runtime.  I guess two options are:
>
> - write a wrapper function around map that communicates with some kind
> of scheduler process   (presumably not the build-in scheduler, at
> least initially), to split the map across multiple cores if there are
> some available
>
> - extend the vm (and perhaps the byte-code) so that the vm can detect
> maps and automatically split them across multiple available processor
> cores where appropriate
>
> Just kindof contemplating out of curiosity.  No idea how feasible it
> is, or how useful, but interests me, especially with Tilera-64s and so
> on on the horizon.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>