[erlang-questions] discrepancy of memory usage figures

[Juan Jose Comellas]

Have you tried the #erlang IRC channel on Freenode? You'll probably find a
lot of Erlang developers there.

Juanjo


On Thu, Jun 23, 2011 at 10:57 PM, Andy W. Song <wsongcn@REDACTED> wrote:

> Thanks for the suggestions. I left the machine for more than 10 hours and
> the figures didn't change.
>
> I'm out of ideas. How can I approach to experts in stead
> of passively waiting on the mailing list?
>
> Thanks
> Andy
>
>
> On Thu, Jun 23, 2011 at 8:38 PM, Jesper Louis Andersen <
> jesper.louis.andersen@REDACTED> wrote:
>
>> On Thu, Jun 23, 2011 at 05:39, Andy W. Song <wsongcn@REDACTED> wrote:
>>
>> > The figures in red are provided htop, others are output of memory() call
>> > from erlang shell. Please look at the total and erlang residence memory.
>> > When there is no connection, these two are roughly same, with 100K
>> > connections, residence memory is a little larger than total, with 200K
>> > connections, residence memory is almost double the total.
>> > Can anybody explain?
>>
>> The first thing I would analyze in this situation is memory
>> fragmentation obtained from the underlying memory allocation. If the
>> memory fragments, then surely you can see stuff like this happening
>> where your resident set size will be larger. I would definitely start
>> out by setting the spotlight on the underlying allocator or the
>> allocation pattern because as you see, there is no discrepancy if you
>> ask the Erlang VM about its view on the matter. What speaks against
>> fragmentation is the very young life of the program. Fragmentation
>> problems usually only show up in long-running programs where pointers
>> are set in stone (making compaction impossible).
>>
>> Another option is that there is some auxiliary data structure in the
>> Erlang VM which are grown up in between the 100K and 200K connection
>> count. If the structure is accessed randomly, then surely all the
>> Virtual Memory pages will get allocated and assigned to the process,
>> increasing the RSS of the program.
>>
>> It is also interesting to look for what happens if you simply ignore
>> the program for a bit of time. Perhaps the allocated memory is not
>> released right away, but over time.
>>
>> Apart from my ideas, I am afraid you will need an Erlang VM expert
>> with more intrinsic knowledge than me.
>>
>>
>> --
>> J.
>>
>
>
>
> --
> ---------------------------------------------------------------
> 有志者，事竟成，破釜沉舟，百二秦关终属楚
> 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110624/1b503264/attachment.htm>