[erlang-questions] Relationship between erlang:memory/1, ets:info/2 and erlang:process_info/2

[Rickard Green]

On Sep 21, 2012, at 6:50 PM, Simon MacMullen wrote:

> I'm trying to write something that will account for memory use on a RabbitMQ system. To do this I'm trying to match up the numbers I get out of erlang:memory(ets) and ets:info(T, memory), and erlang:memory(processes) and erlang:process_info/2.
> 
> And it doesn't make a huge amount of sense to me. I don't expect things to match up perfectly, of course there will be some memory unaccounted for, but the ETS numbers in particular can be quite off:
> 
> 1> erlang:memory(ets).
> 2120291872
> 
> 2> lists:sum([ets:info(T, memory) || T <- ets:all()]) * erlang:system_info(wordsize).
> 1874292776
> 
> So I have ~2GB of ETS memory, and ~230MB unaccounted for. Should I expect these to match up closer?
> 

The main structure of an ets table is erroneously accounted for twice in the erlang:memory(ets) case. :-(

> And then...
> 
> 3> erlang:memory(processes).
> 1519425156
> 
> 4> lists:sum([element(2, process_info(P, memory)) || P <- processes()]). 
> 1552852008
> 
> So for processes there is less in total than if you add them all up. Is some memory shared? (Are we counting binaries and atoms in there as well?)
> 

This can happen when there are a lot of outstanding messages.

> Is there a way to get this sort of information in a more consistent manner?
> 

The ets case above will be fixed. In the process case I haven't decided what to do, yet, but something will be done.

erlang:memory/[0,1] use a very different (and cheaper) strategy than ets:info() and process_info(). The results will more or less always differ, and should since there are memory allocated that isn't for any specific process or ets-table in the erlang:memory() case.

If you want to know memory used for all processes or all ets-tables use erlang:memory(). Much cheaper (both from the callers perspective, and especially from an overall performance perspective), as well as generally a more true value.

Summing up results of process_info(), or ets:info() will, of course, not give you a consistent snapshot of the memory usage at one point in time. Note that the same is true in the erlang:memory() case. The result is the sum of memory usage in different allocator instances that have been read at different (but close) points in time.

Regards,
Rickard Green, Erlang/OTP, Ericsson AB

> Cheers, Simon
> 
> -- 
> Simon MacMullen
> RabbitMQ, VMware
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions