[erlang-questions] slow file I/O (was Not an Erlang fan)
Ulf Wiger (TN/EAB)
Thu Sep 27 09:38:44 CEST 2007
The Haskell code uses functions more similar to
file:read_file/1 and file:read/2, which are not
nearly as slow as io:get_line/2.
I agree with Klacke, that if we could just get flow
control on ports, then there is a perfectly fine
line-oriented mode built into the port. It's so fast,
that using it without flow control on a large file
will swamp your erlang code (I tried that in the shootout,
with disastrous results).
Steve Vinoski wrote:
> The whole discussion surrounding the Erlang issues that Tim Bray
> presented in his blog has got me wondering about file I/O. I'm
> definitely no Erlang expert, but my own experiments seem to show that
> Erlang file I/O is an insurmountable obstacle for the kind of problem
> Tim's trying to solve. Unfortunately, clear details about file I/O
> didn't seem to come out in the "Not an Erlang fan" thread.
> So, I'm wondering:
> 1. Is file I/O for large files really as slow as it seems, and if so, why?
> 2. Are there existing alternatives to the regular file module functions
> for file I/O that might skirt this problem, and if so, what are they?
> 3. Is the whole premise of this problem just not "how it's done" in
> Erlang? If so, how would this problem be rearranged to better allow for
> an efficient Erlang solution on the large dataset?
> 4. Someone posted a link to a Haskell solution in a comment in my blog
> that seems too good to be true:
> Assuming it's accurate, why does Haskell beat Erlang so handily in this
> 5. If file I/O speed is really the issue that it seems to be, are there
> any plans to officially fix it?
> erlang-questions mailing list
More information about the erlang-questions