[erlang-questions] widefinder update

[Hynek Vychodil]

>
> Currently that sequential part is ~ 0.5s on my 1.66GHz
> dual core laptop.
> the part of the work that can be run in parallel takes
> ~2.254 s
> so theoretically we would get
> Cores  Real time  Speedup  Rel. speedup by doubling #cores
> 1         2.754
> 2         1.627       1.693      1.693
> 4         1.064       2.590      1.530
> 8         0.782       3.523      1.360
> 16       0.641       4.297      1.220
> 32       0.570       4.828      1.123
> 64       0.535       5.146      1.066
> 128     0.518       5.321      1.034
> 256     0.509       5.413      1.017
>
> Which is not very good after 8 cores.
>
> So I am now looking at making this a 'real' distributed solution instead.
>
> /Anders

I wrote some code to try parallel splitting, but it is still with
single dispatcher, because you must have single reader. Sequential
reading is about twenty times faster than random and you must keep
information about block order. Let you see chunk_reader and
nlt_reader.

Crazy think that chunk_reader is slower when compiled with native
option. I don't know why.

-- Hynek (Pichi) Vychodil
-------------- next part --------------
A non-text attachment was scrubbed...
Name: chunk_reader.erl
Type: application/octet-stream
Size: 3722 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071028/16fc8af3/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nlt_reader.erl
Type: application/octet-stream
Size: 7985 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071028/16fc8af3/attachment-0001.obj>