[erlang-questions] comment on my erlang Spamfilter

[James Hague]

readfile(FileName) ->
    {ok, Binary} = file:read_file(FileName),
    string:tokens(binary_to_list(Binary), " ").

Were I writing this, I wouldn't have called string:tokens at all, but
directly looped through Binary looking for words.  More than once I've
found string:tokens to be a hotspot, and there's no compelling reason
to do the tokenization as a separate step in this case.  That also
avoids the 8x blowup caused by binary_to_list on a potentially large
file.