[erlang-questions] Strange performance issue - advice needed

Ian <>
Sat Apr 14 21:56:06 CEST 2012

Thanks Tim,

Before you posted I had tried a version that wrote each word to the file 
as it arrived.  That took 3550 seconds!

So I rewrote the code as below and it runs in 800ms at 50 chars line 
length and 769ms for 200 chars per line - much better and more the 
relationship I expected. Thanks for your helpful input.

But, my goodness, so called buffered file IO is inexpressibly crap!



On 14/04/2012 18:14, Gmail wrote:
> Can't read the code well on my phone but if you're using line oriented io then that tends to suck.
> I would suggest reading http://erlang.org/pipermail/erlang-questions/2012-March/065374.html as well.
> Cheers,
> Tim
Replacement code - smaller and 20 times faster (and needs a bit more RAM).

%% tgpack - read words, builds and write lines to FileName,
%% where lines are as long as possible and less than Length
pack(Length,FileName) ->
     file:write_file(FileName, buildlist(Length,0,[])).

% buildList
% Length is max length of line
% Size is length of line so far
% List is list of binaries built up so far, in reverse order
buildlist(Length,Size,List) ->
<<>> ->
         Word ->
             S = Size + 1 + byte_size(Word), % length of current line
             if S >= Length ->  % start new line
             true ->
                 buildlist(Length,S,[Word|[<<" ">>|List]])

More information about the erlang-questions mailing list