[erlang-questions] discrepancy of memory usage figures

[Andy W. Song]

Thanks for the suggestions. I left the machine for more than 10 hours and
the figures didn't change.

I'm out of ideas. How can I approach to experts in stead
of passively waiting on the mailing list?

Thanks
Andy

On Thu, Jun 23, 2011 at 8:38 PM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

> On Thu, Jun 23, 2011 at 05:39, Andy W. Song <wsongcn@REDACTED> wrote:
>
> > The figures in red are provided htop, others are output of memory() call
> > from erlang shell. Please look at the total and erlang residence memory.
> > When there is no connection, these two are roughly same, with 100K
> > connections, residence memory is a little larger than total, with 200K
> > connections, residence memory is almost double the total.
> > Can anybody explain?
>
> The first thing I would analyze in this situation is memory
> fragmentation obtained from the underlying memory allocation. If the
> memory fragments, then surely you can see stuff like this happening
> where your resident set size will be larger. I would definitely start
> out by setting the spotlight on the underlying allocator or the
> allocation pattern because as you see, there is no discrepancy if you
> ask the Erlang VM about its view on the matter. What speaks against
> fragmentation is the very young life of the program. Fragmentation
> problems usually only show up in long-running programs where pointers
> are set in stone (making compaction impossible).
>
> Another option is that there is some auxiliary data structure in the
> Erlang VM which are grown up in between the 100K and 200K connection
> count. If the structure is accessed randomly, then surely all the
> Virtual Memory pages will get allocated and assigned to the process,
> increasing the RSS of the program.
>
> It is also interesting to look for what happens if you simply ignore
> the program for a bit of time. Perhaps the allocated memory is not
> released right away, but over time.
>
> Apart from my ideas, I am afraid you will need an Erlang VM expert
> with more intrinsic knowledge than me.
>
>
> --
> J.
>



-- 
---------------------------------------------------------------
有志者，事竟成，破釜沉舟，百二秦关终属楚
苦心人，天不负，卧薪尝胆，三千越甲可吞吴
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110624/87043652/attachment.htm>