[erlang-questions] Max open files and erlang memory

[Per Hedeland]

Vincent de Phily <vincent.dephily@REDACTED> wrote:
>
>I understand that opening more files uses more memory, but I haven't opened 
>anything yet. I shouldn't be paying the price for a system limit I haven't 
>reached yet.
>Also, note that the memory usage in the above example is not linear : there 
>are jumps in the graph.
>
>I am not 100% sure, but I think this is a regression : I tested my software 
>with a big max-open-file limit before and hadn't noticed the memory usage.

The runtime creates an array of port structures (100+ bytes IIRC) sized
as the next power of two at or above your "max-open-files" (a.k.a.
sysconf(_SC_OPEN_MAX)) - this has been true since at least R10, probably
much earlier. I.e. the following from erlang(3) is only correct
regarding the default if you have a "max-open-files" of 1024 (or less) -
which happens to be the default on Linux I believe.

      The maximum number of ports that can be open at the  same  time
      is  1024  by  default,  but can be configured by the environment
      variable ERL_MAX_PORTS.

The ERL_MAX_PORTS part *is* correct though - and you can use this to
*reduce* the size of that port array. Oh, there is actually an upper
limit - 2^28 as far as I can see. And 1024 is the *lower* limit. See
init_io() in erts/emulator/beam/io.c.

--Per Hedeland