[erlang-questions] A more parallel widefinder

[Anders Nygren]

One more try.
This one tries to do more in the workers and as little as possible in
the central controller.

The controller only
- starts workers, and maintains a specific number of workers.
- prints the results

Each worker
- opens the file and reads a chunk
- finds the first and last newlines in the chunk
- sends the partial line at the beginning of the chunk
to the worker processing the previous chunk
- processes its chunk, excluding the initial and final
partial lines
- receives a partial line from from the worker that handles
the next chunk
- concatenates and process the last line

The sequential control part now takes ~180 ms
A worker processing a 200kbytes chunk takes ~1.7 ms
So it will scale better to more cores than the previous versions.

I estimate that on Steve's 8 core it will run at ~ 0.3 s.

It is actually slightly faster than my previous wfinder1_1 even
on a dual core.

/Anders
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfinder3.erl
Type: text/x-erlang
Size: 4218 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071029/e6c9acdf/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfbm4_ets2.erl
Type: text/x-erlang
Size: 3520 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071029/e6c9acdf/attachment-0001.bin>