[erlang-questions] Memory consuption: what is 'system' an how to compact

[Danil Zagoskin]

Hi!

Our application has allocated over 11 GB of memory and I don't understand
why.

The application contains:
 - HTTPS server using old mochiweb and OTP SSL
 - interfaces to internal services using lhttpc
 - caches using ETS

First problem: over 11 GB is allocated but only 5 GB are used:
> erlang:memory().
[{total,5154952544},
 {processes,1893901080},
 {processes_used,1893832024},
 {system,3261051464},
 {atom,553569},
 {atom_used,533811},
 {binary,520767552},
 {code,12580030},
 {ets,154178408}]

I understand that 50% allocator usage ratio is subject to changing
allocation strategy.

Question: Are there any tips on choosing one apart of "try and see what
happens"?


Second problem: as you can see above, system consumes 3.2 GB when atom,
binary, code, ets sum up to less than 1 GB.

Question: Where did 2.5 GB go? Earlier (R16B+) this application could run
for weeks with half (13K) of current (28K) connection count and use less
than 2 GB. The most significant change about memory is a small binary kept
in each worker's dictionary.


Third problem:
MBCSs = [proplists:get_value(mbcs_usage, Info) || {{binary_alloc,_}, Info}
<- recon_alloc:fragmentation(current)], lists:sum(MBCSs)/length(MBCSs).
0.13687142195007596

Question: is there a way to reallocate long-living binaries without
changing the code so that they don't hold this much memory?
Question: How can I list binaries in single carrier? I want to see contents
and referencing pids.
Question: Given an entry from process_info(Pid, binary) like
this: {139767104723144,19,1} how can I see carrier ID and contents of this
binary?


I have recon_alloc snapshot and can send it if needed.

-- 
Danil Zagoskin | z@REDACTED
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140722/124a1b3b/attachment.htm>