[erlang-questions] os_mon memsup reporting memory consumption

[Serge Aleynikov]

Hi Gunilla,

Gunilla Arendt wrote:
> I see the point, but it would be a non-backward compatible change
> and as you know we're very hesitant about those.

I would imagine that if anyone used OS_MON & SASL on Linux in production 
they would quickly run into this issue, and if so, there would be some 
posts about it.  Also, if this fix was available in a future version of 
OTP with appropriate compatibility note, I would assume that any user 
who would want to upgrade a production system from a prior OTP release 
to that one would read the compatibility notes...

The only impact I would think of would be that if a production system 
running on Linux had the OS_MON's system_memory_high_watermark option 
set to something higher than default (say 95%) to compensate for false 
positive alarm behavior (when in reality memory consumption might be 
50%), introduction of this fix without modifying the 
system_memory_high_watermark environment could potentially produce an 
alarm when the system's memory consumption is indeed at 95% that may be 
a too severe threshold for the system.

I don't know what others on the list feel about this oddity, but setting 
an option to 95% in hope that it would indicate a 50% threshold is 
confusing to say the least.

> An alternative solution could be to add a new tag to the result of
> memsup:get_system_memory_data() for Linux systems, although that would
> not affect the setting/clearing of alarms.

I believe the real benefit of OS_MON comes when it produces meaningful 
O&M alarms.  If it doesn't the value of the app diminishes by quite a bit.

Regards,

Serge

> Serge Aleynikov wrote:
>> We had a few high memory watermark alarms in a production system running 
>> Linux reported by OS_MON whereas it seemed like the memory utilization 
>> on the server wasn't bad.  I looked at the implementation of the 
>> memsup.erl module in OS_MON, and I believe the calculation of allocated 
>> memory has a flow.
>>
>> Memsup's code (sited at the end) reveals that allocated memory doesn't 
>> take in consideration cached memory.  Since cached memory can be quickly 
>> freed by the Linux memory manager if additional memory is needed by 
>> applications, it looks to me that it should be:
>>
>> Allocated = MemTotal - (MemFree + Cached)
>>
>> or even:
>>
>> Allocated = MemTotal - (MemFree + Buffers + Cached)
>>
>> This would be consistent with what "free" is showing on the "-/+ 
>> buffers/cache" line:
>>
>> drp@REDACTED: ~$ uname -a
>> Linux drpdb02.corp.idt.net 2.6.9-5.ELsmp #1 SMP Wed Jan 5 19:30:39 EST 
>> 2005 i686 i686 i386 GNU/Linux
>>
>> drp@REDACTED: ~$ free
>>              total      used       free     shared    buffers     cached
>> Mem:       4150324   3416788     733536          0     282412    2621788
>> -/+ buffers/cache:    512588    3637736
>> Swap:      2048248         0    2048248
>>
>> drp@REDACTED: ~$ head -4 /proc/meminfo
>> MemTotal:      4150324 kB
>> MemFree:        733536 kB
>> Buffers:        282412 kB
>> Cached:        2621788 kB
>>
>>
>>
>> %-------------------------
>> %% @spec get_memory_usage_linux() -> {Allocated, Total}
>> %-------------------------
>> get_memory_usage_linux() ->
>>     Res = os:cmd("cat /proc/meminfo"),
>>     case get_memory_usage_linux(Res, undef, undef) of
>>     {MemTotal, MemFree} ->
>>         {MemTotal-MemFree, MemTotal};
>>     error ->
>>         timer:sleep(1000),
>>         get_memory_usage_linux()
>>     end.
>>
>> get_memory_usage_linux(_Str, Tot, Free) when is_integer(Tot),
>>                          is_integer(Free) ->
>>     {Tot, Free};
>> get_memory_usage_linux("MemTotal:"++T, _Tot0, Free0) ->
>>     {ok, [N], Rest} = io_lib:fread("~d", T),
>>     Tot = N*1024,
>>     get_memory_usage_linux(skip_to_eol(Rest), Tot, Free0);
>> get_memory_usage_linux("MemFree:"++T, Tot0, _Free0) ->
>>     {ok, [N], Rest} = io_lib:fread("~d", T),
>>     Free = N*1024,
>>     get_memory_usage_linux(skip_to_eol(Rest), Tot0, Free);
>> get_memory_usage_linux("", _Tot0, _Free0) ->
>>     error;
>> get_memory_usage_linux(Str, Tot0, Free0) ->
>>     get_memory_usage_linux(skip_to_eol(Str), Tot0, Free0).
>> %-------------------------
>>
>> Regards,
>>
>> Serge
>>
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 


-- 
Serge Aleynikov
R&D Telecom, MIS, IDT Corp
Tel: +1 (973) 438-3436
Fax: +1 (973) 438-1464