[erlang-questions] discrepancy of memory usage figures
Juan Jose Comellas
Fri Jun 24 15:32:51 CEST 2011
Have you tried the #erlang IRC channel on Freenode? You'll probably find a
lot of Erlang developers there.
On Thu, Jun 23, 2011 at 10:57 PM, Andy W. Song <wsongcn@REDACTED> wrote:
> Thanks for the suggestions. I left the machine for more than 10 hours and
> the figures didn't change.
> I'm out of ideas. How can I approach to experts in stead
> of passively waiting on the mailing list?
> On Thu, Jun 23, 2011 at 8:38 PM, Jesper Louis Andersen <
> jesper.louis.andersen@REDACTED> wrote:
>> On Thu, Jun 23, 2011 at 05:39, Andy W. Song <wsongcn@REDACTED> wrote:
>> > The figures in red are provided htop, others are output of memory() call
>> > from erlang shell. Please look at the total and erlang residence memory.
>> > When there is no connection, these two are roughly same, with 100K
>> > connections, residence memory is a little larger than total, with 200K
>> > connections, residence memory is almost double the total.
>> > Can anybody explain?
>> The first thing I would analyze in this situation is memory
>> fragmentation obtained from the underlying memory allocation. If the
>> memory fragments, then surely you can see stuff like this happening
>> where your resident set size will be larger. I would definitely start
>> out by setting the spotlight on the underlying allocator or the
>> allocation pattern because as you see, there is no discrepancy if you
>> ask the Erlang VM about its view on the matter. What speaks against
>> fragmentation is the very young life of the program. Fragmentation
>> problems usually only show up in long-running programs where pointers
>> are set in stone (making compaction impossible).
>> Another option is that there is some auxiliary data structure in the
>> Erlang VM which are grown up in between the 100K and 200K connection
>> count. If the structure is accessed randomly, then surely all the
>> Virtual Memory pages will get allocated and assigned to the process,
>> increasing the RSS of the program.
>> It is also interesting to look for what happens if you simply ignore
>> the program for a bit of time. Perhaps the allocated memory is not
>> released right away, but over time.
>> Apart from my ideas, I am afraid you will need an Erlang VM expert
>> with more intrinsic knowledge than me.
> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions