[erlang-questions] Strange performance issue - advice needed

[Ian]

Thanks Tim,

Before you posted I had tried a version that wrote each word to the file 
as it arrived.  That took 3550 seconds!

So I rewrote the code as below and it runs in 800ms at 50 chars line 
length and 769ms for 200 chars per line - much better and more the 
relationship I expected. Thanks for your helpful input.

But, my goodness, so called buffered file IO is inexpressibly crap!

Regards

Ian

On 14/04/2012 18:14, Gmail wrote:
> Can't read the code well on my phone but if you're using line oriented io then that tends to suck.
>
> I would suggest reading http://erlang.org/pipermail/erlang-questions/2012-March/065374.html as well.
>
> Cheers,
> Tim
>
Replacement code - smaller and 20 times faster (and needs a bit more RAM).

-module(tgpack).
-export([pack/2]).
%% tgpack - read words, builds and write lines to FileName,
%% where lines are as long as possible and less than Length
pack(Length,FileName) ->
     file:write_file(FileName, buildlist(Length,0,[])).

% buildList
% Length is max length of line
% Size is length of line so far
% List is list of binaries built up so far, in reverse order
buildlist(Length,Size,List) ->
     receive
<<>> ->
             lists:reverse(List);
         Word ->
             S = Size + 1 + byte_size(Word), % length of current line
             if S >= Length ->  % start new line
                 buildlist(Length,byte_size(Word),[Word|[<<"\n">>|List]]);
             true ->
                 buildlist(Length,S,[Word|[<<" ">>|List]])
             end
     end.