[erlang-questions] Relationship between erlang:memory/1, ets:info/2 and erlang:process_info/2

Rickard Green rickard@REDACTED
Mon Sep 24 01:34:45 CEST 2012

On Sep 21, 2012, at 6:50 PM, Simon MacMullen wrote:

> I'm trying to write something that will account for memory use on a RabbitMQ system. To do this I'm trying to match up the numbers I get out of erlang:memory(ets) and ets:info(T, memory), and erlang:memory(processes) and erlang:process_info/2.
> And it doesn't make a huge amount of sense to me. I don't expect things to match up perfectly, of course there will be some memory unaccounted for, but the ETS numbers in particular can be quite off:
> 1> erlang:memory(ets).
> 2120291872
> 2> lists:sum([ets:info(T, memory) || T <- ets:all()]) * erlang:system_info(wordsize).
> 1874292776
> So I have ~2GB of ETS memory, and ~230MB unaccounted for. Should I expect these to match up closer?

The main structure of an ets table is erroneously accounted for twice in the erlang:memory(ets) case. :-(

> And then...
> 3> erlang:memory(processes).
> 1519425156
> 4> lists:sum([element(2, process_info(P, memory)) || P <- processes()]). 
> 1552852008
> So for processes there is less in total than if you add them all up. Is some memory shared? (Are we counting binaries and atoms in there as well?)

This can happen when there are a lot of outstanding messages.

> Is there a way to get this sort of information in a more consistent manner?

The ets case above will be fixed. In the process case I haven't decided what to do, yet, but something will be done.

erlang:memory/[0,1] use a very different (and cheaper) strategy than ets:info() and process_info(). The results will more or less always differ, and should since there are memory allocated that isn't for any specific process or ets-table in the erlang:memory() case.

If you want to know memory used for all processes or all ets-tables use erlang:memory(). Much cheaper (both from the callers perspective, and especially from an overall performance perspective), as well as generally a more true value.

Summing up results of process_info(), or ets:info() will, of course, not give you a consistent snapshot of the memory usage at one point in time. Note that the same is true in the erlang:memory() case. The result is the sum of memory usage in different allocator instances that have been read at different (but close) points in time.

Rickard Green, Erlang/OTP, Ericsson AB

> Cheers, Simon
> -- 
> Simon MacMullen
> RabbitMQ, VMware
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

More information about the erlang-questions mailing list