[erlang-questions] slow file I/O (was Not an Erlang fan)

[Ulf Wiger (TN/EAB)]

The Haskell code uses functions more similar to
file:read_file/1 and file:read/2, which are not
nearly as slow as io:get_line/2.

I agree with Klacke, that if we could just get flow
control on ports, then there is a perfectly fine
line-oriented mode built into the port. It's so fast,
that using it without flow control on a large file
will swamp your erlang code (I tried that in the shootout,
with disastrous results).

http://www.erlang.org/pipermail/erlang-questions/2007-June/027557.html

BR,
Ulf W

Steve Vinoski wrote:
> The whole discussion surrounding the Erlang issues that Tim Bray 
> presented in his blog has got me wondering about file I/O. I'm 
> definitely no Erlang expert, but my own experiments seem to show that 
> Erlang file I/O is an insurmountable obstacle for the kind of problem 
> Tim's trying to solve. Unfortunately, clear details about file I/O 
> didn't seem to come out in the "Not an Erlang fan" thread.
> 
> So, I'm wondering:
> 
> 1. Is file I/O for large files really as slow as it seems, and if so, why?
> 2. Are there existing alternatives to the regular file module functions 
> for file I/O that might skirt this problem, and if so, what are they?
> 3. Is the whole premise of this problem just not "how it's done" in 
> Erlang? If so, how would this problem be rearranged to better allow for 
> an efficient Erlang solution on the large dataset?
> 4. Someone posted a link to a Haskell solution in a comment in my blog 
> that seems too good to be true: 
> <http://www.serpentine.com/blog/2007/09/25/what-the-heck-is-a-wide-finder-anyway/ 
> <http://www.serpentine.com/blog/2007/09/25/what-the-heck-is-a-wide-finder-anyway/>>. 
> Assuming it's accurate, why does Haskell beat Erlang so handily in this 
> situation?
> 5. If file I/O speed is really the issue that it seems to be, are there 
> any plans to officially fix it?
> 
>  
> thanks,
> --steve
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions