[erlang-questions] Not an Erlang fan

[Steve Vinoski]

On 9/23/07, Bob Ippolito <bob@REDACTED> wrote:
>
> On 9/24/07, Patrick Logan <patrickdlogan@REDACTED> wrote:
> > > > >  http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang
> > > > >
> > > > > Tim Bray might raise some valid points here, even if he's slightly
> > > > > biased by his background.
> >
> > The good news is speeding up the i/o in erlang should be easier than
> > introducing better concurrency to another language.
> >
>
> I've never had a problem with Erlang's general I/O performance, it's
> probably just some implementation detail of direct file I/O that is
> the loser here. The obvious Erlang fast path to read lines is to spawn
> cat and let the port machinery do all of the work for you. Here's an
> example (including a copy of Tim's dataset):
>
> http://undefined.org/erlang/o10k.zip
>

I posted a link in a comment to Tim's blog to an example that uses multiple
processes to break down the expensive parts of processing Tim's dataset in
parallel, and was able to achieve a pure Erlang approach that on my MacBook
Pro equals your "cat" approach, and is much faster than "cat" on an 8-core
machine. It's shown on my blog:

<http://steve.vinoski.net/blog/2007/09/23/tim-bray-and-erlang/>

It definitely speeds up as the number of cores goes up.

I don't consider myself an Erlang expert and so welcome any suggestions for
improving this. I'm guessing someone will see the two instances of "++" list
handling and jump on that, but I tried it with the typical reverse approach
and with flattening and neither was faster. However I am quite open to being
enlightened. :-)

--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070924/79f0b29f/attachment.htm>