[erlang-questions] Strange performance issue - advice needed
Sat Apr 14 21:56:06 CEST 2012
Before you posted I had tried a version that wrote each word to the file
as it arrived. That took 3550 seconds!
So I rewrote the code as below and it runs in 800ms at 50 chars line
length and 769ms for 200 chars per line - much better and more the
relationship I expected. Thanks for your helpful input.
But, my goodness, so called buffered file IO is inexpressibly crap!
On 14/04/2012 18:14, Gmail wrote:
> Can't read the code well on my phone but if you're using line oriented io then that tends to suck.
> I would suggest reading http://erlang.org/pipermail/erlang-questions/2012-March/065374.html as well.
Replacement code - smaller and 20 times faster (and needs a bit more RAM).
%% tgpack - read words, builds and write lines to FileName,
%% where lines are as long as possible and less than Length
% Length is max length of line
% Size is length of line so far
% List is list of binaries built up so far, in reverse order
S = Size + 1 + byte_size(Word), % length of current line
if S >= Length -> % start new line
More information about the erlang-questions