[erlang-questions] A more parallel widefinder
Tue Oct 30 16:58:54 CET 2007
On 10/30/07, Hynek Vychodil <> wrote:
> Good work. It's very nice idea to find last new line in chunk and than
> not concatenate binary with big prefix. But I still argue, that this
> implementation is not so much like universal tool, but especial one
> purpose tuned solution. I'm so busy to try rewrite it to more modular
> implementation. But I will try it some time.
I think the idea can have universal use, for the class of data where
it is possible to start scanning at an arbitrary position and identify
the start or end of a logical data unit.
Scanning the entire file to identify the block separators creates a large
sequential term that can not take advantage to more cores.
More information about the erlang-questions