multi-core processors and erlang

[Joe Armstrong (TN/EAB)]

Well the code I was referring to was a wiki markup language -> Erlang
rich text -> html
transformation.

Now this is written in a horrendously inefficient style - I think I take
about five separate passes
though the data to make the rich text, with some careful thought this
program 
could be turned into a single pass nightmare that was more efficient and
impossible to maintain.

Most program optimisations take place in my head before the program ever
gets written,
I try to choose reasonable representations that are a balance between
clarity, speed, and.

Usually by concentrating on clarity, and a decent representation, speed
will follow.

When you actually get round to writing the stuff, I know when to use
binaries, or lists
or trees or ets tables etc - but it's very difficult to give rules for
this - all ya need to do is
spend twenty years writing thousands of modules and it gets pretty easy
:-)  
 
What I liked about the example I posted (and the point of the article)
was that
the top level pseudo code for my problem was:

	foreach file in Dir do 
		process(file)

Given that the processing of the files, could de done simultaneously,
the top level
code became

	foreach file in Dir do IN PARALLEL
		process(file)

This in practise meant replacing a single map, with a pmap and I had to
write a (trivial) pmap
function.

Now we need some care in replacing maps with pmaps, so replacing

	map(fun(I) -> compile_file(I) end, L) with a pmap would be fine,
but

	map(fun(I) -> 2*I end, L) would be silly since the effort in
computing 2*I is
less than the effort required to spawn #L processes and gather the
results.

/Joe



> -----Original Message-----
> From: Inswitch Solutions [mailto:erlang@REDACTED] 
> Sent: den 20 juli 2006 14:11
> To: Joe Armstrong (TN/EAB); Only OpenSource; 
> erlang-questions@REDACTED
> Subject: Re: multi-core processors and erlang
> 
> Hi Joe,
> 
> In the link
> http://www.erlang.org/ml-archive/erlang-questions/200606/msg00
> 187.html  you
> wrote:
> "Sure I *could* speed things up with messy optimised code, 
> but the code would be less beautiful, and more difficult to maintain."
> 
> What kind of optimizations can be done, to the code mentioned 
> there, to speed things up?



> 
> 
> regards, Eduardo Figoli
> 
> 
> 
> ----- Original Message -----
> From: "Joe Armstrong (TN/EAB)" <joe.armstrong@REDACTED>
> To: "Only OpenSource" <onlyopensource@REDACTED>; 
> <erlang-questions@REDACTED>
> Sent: Thursday, July 20, 2006 6:15 AM
> Subject: RE: multi-core processors and erlang
> 
> 
> >
> > Absolutely -
> >
> > <<aside>>
> > just think, when "other languages" (TM) have caught onto the idea
> > of "no sharing, no-locking, lightweight processes" they will be were
> > Erlang was twenty years ago :-)
> > <</aside>>
> >
> > Yes Erlang runs happily on multi core processors - drop your Erlang
> > program onto a multi core
> > and it *will* go faster, exactly how much faster depends upon the
> > dependencies between you parallel
> > processes. If they are all independent and CPU bound you'll 
> probably run
> > K * N
> > times faster on an N core CPU. Say 16 times on a Sun Niagara.
> >
> > << K depends upon the Erlang implementation, it's currently 
> about 0.5,
> > were hoping for 0.75,
> >    the current implementation strategy emphasis stability 
> over speed ...
> > >>
> >
> > Turning a sequential program into a parallel program is also really
> > really easy, for
> > example, I changed a single map to pmap (pmap is a parallel map
> > function)
> > and the program went 7 times faster.
> >
> > Read
> >
> > 
> http://www.erlang.org/ml-archive/erlang-questions/200606/msg00187.html
> >
> >
> > See also
> >
> > Multicore Erlang performance
> >
> > http://www.erlang-stuff.net/blog/
> >
> >
> > /Joe
> >
> > > -----Original Message-----
> > > From: owner-erlang-questions@REDACTED
> > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Only
> > > OpenSource
> > > Sent: den 20 juli 2006 10:52
> > > To: erlang-questions@REDACTED
> > > Subject: multi-core processors and erlang
> > >
> > > hello
> > >
> > > Does erlang take benefit of multi-core processors which are
> > > now available in desktop machines ?
> > >
> > > erlang supports concurrency and can definitely benefit from
> > > native support from multi-core processors.
> > >
> > > Is it a correct assumption or am I just too sleepy ?
> > >
> > > --
> > > oo@@oo
> > >
> >
> 
> 
>