The real reason why file:read_file in R7B guzzles memory

[matthias@REDACTED]

Hi,

Miguel Barreiro observed that file:read_file/1 seemed to be
unexpectedly memory hungry.

It has nothing to do with the history list, nor is it the natural
consequence of printing. It's a bug in the io library.

Matthias

----------------------------------------------------------------------

Reproducing the bug

   1> {ok, Bin} = file:read_file("/usr/share/dict/words"), z.
   2> io_lib:fwrite("~P", [Bin, 7]), z. 
   3> io_lib:fwrite("~P", [{ok, Bin}, 7]), z.

   line 3 takes an unreasonably long time and an unreasonable amount
   of memory, while line 2 does not. (So Martin is technically right
   when he says it's because we're printing the binary---though
   the real problem is that we're printing the _whole_ binary)

Removing the bug

   One way is to change the clause in io_lib_pretty:write_length 
   for binaries to

    write_length(Bin, D, Acc, Max) when binary(Bin) ->
       Acc;

   I'm not sure what the 'depth' rules are, but the result
   looks reasonable. The old clause was

    write_length(Bin, D, Acc, Max) when binary(Bin) ->
       Acc + length(io_lib:write(Bin));

   This performs badly because it causes io_lib:write/1 to
   generate a list representing the whole binary. In R6 it
   performed ok because binaries were always written as a short
   string.