[erlang-bugs] erts_debug:flat_size/1 wrong?

[Michael Truog]

On 09/26/2014 11:27 AM, Michael Truog wrote:
> On 09/26/2014 06:46 AM, Sverker Eriksson wrote:
>> On 09/26/2014 04:19 AM, Michael Truog wrote:
>>> Hi,
>>>
>>> I have been attempting to compare the output of erts_debug:flat_size/1 to the memory info at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923 and the results show that each term's size is off-by-one (at least for pids local/remote, refs local/remote, floats, integers, bignums, binaries and atoms).  I know the function is experimental, but this is a bug, right?  The problem affects top-level terms and nested terms, so it is likely to understate the memory with large terms.  I wanted to make sure the memory info (in the efficiency guide) was accurate (it seems like it is).  I was testing with R16B03 on 64bits.
>>>
>>> For example:
>>> 1> erts_debug:flat_size(576460752303423488).
>>> 2
>>> 2> erts_debug:flat_size(576460752303423487).
>>> 0
>>> 3> erts_debug:flat_size(undefined).
>>> 0
>>> 4> erts_debug:flat_size([]).
>>> 0
>>> 5> erts_debug:flat_size([undefined]).
>>> 2
>>> % 1 word for each element in the list * 2 elements including []
>>> 6> erts_debug:flat_size(erlang:make_ref()).
>>> 3
>>> 7> erts_debug:flat_size(erlang:self()).
>>> 0
>>> 8> erts_debug:flat_size(1.0).
>>> 2
>>>
>>
>>
>> erts_debug:flat_size/1 is undocumented and can therefor not be wrong by definition :-)
>>
>> But no, there is no bug, erts_debug:flat_size/1 works as intended in the current implementation. It returns the number of words on the _heap_ occupied (*) by the term. Excluded are thus off-heap data such as binaries larger than 64 bytes AND the top term word, which is kept in a register or on the stack.
>>
>> (*) erts_debug:flat_size does not take sharing of sub-terms into account while erts_debug:size/1 do.
>>
>> 1> A = "Hello".
>> "Hello"
>> 2> erts_debug:flat_size(A).
>> 10
>> 3> erts_debug:flat_size({A,A}).
>> 23
>> 4> erts_debug:size({A,A}).
>> 13
>
> Thank you for the information.  However, based on this the Efficiency Guide memory information should be slightly wrong (at http://www.erlang.org/doc/efficiency_guide/advanced.html#id68923), since the information should be
> * List: 1 word + |2 word per element| + the size of each element
> * Tuple: 2 words + |1 word per element| + the size of each element
>
> and not what it currently is:
> * List: 1 word + |1 word per element| + the size of each element
> * Tuple: 2 words + the size of each element
>
> That is at least what I have found when using erts_debug:flat_size/1 in the repo https://github.com/okeuday/erlang_term when looking at the total size of an Erlang term.  Do you agree?
After looking at this more I have realized the documentation of the memory information is correct as would be expected.  Sorry for the noise about this.  Some comment that talks about erts_debug:flat_size/1 (and erts_debug:size/1) providing the process heap size only, with an additional 1 word excluded for the register or stack storage of the top-level term would help make things clearer.  This may be straight-forward for some since it makes logical sense, but I didn't know about these internal details and I wanted to be sure I was looking at the size correctly.

Thanks,
Michael


>
> Thanks,
> Michael
>>
>>
>> /Sverker, Erlang/OTP
>>
>>
>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20140929/843e2dd0/attachment.htm>