[erlang-questions] ETS and CPU

[Alex Howle]

I'm on a 64-bit machine, and flat_size returns the number of words. So I
took the result and multiplied by 8 to get the number of bytes.
On 16 Mar 2016 16:18, "Hynek Vychodil" <vychodil.hynek@REDACTED> wrote:

> Just to be sure, when you wrote that erts_debug:flat_size/1 approximate
> the map's size to be 1MB, you mean something around this value
>
> > erts_debug:flat_size(Map)
> 131072
>
> Because if you have something like
> > erts_debug:flat_size(Map)
> 1048576
>
> It is 8MB and your memory IO is like 25*9/s*8MB = 1800MB/s. It is still
> manageable by your HW but should be considered in your design.
>
> On Wed, Mar 16, 2016 at 10:43 AM, Alex Howle <itshowlertime@REDACTED>
> wrote:
>
>> Assuming that when you say "win" you mean that ets:lookup should be more
>> efficient (and less CPU intensive) then I'm seeing the opposite.
>> On 15 Mar 2016 11:32, "Sverker Eriksson" <sverker.eriksson@REDACTED>
>> wrote:
>>
>>> Each successful ets:lookup call is a copy operation of the entire term
>>> from ETS to the process heap.
>>>
>>> If you are comparing ets:lookup of big map
>>> to sending big map in message then I would expect
>>> ets:lookup to win, as copy_shallow (used by ets:lookup)
>>> is optimized to be faster than copy_struct (used by send).
>>>
>>>
>>> /Sverker, Erlang/OTP
>>>
>>>
>>> On 03/15/2016 09:52 AM, Alex Howle wrote:
>>>
>>> I've been experiencing an issue and was wondering if anyone else has any
>>> experience in this area. I've stripped back the problem to its bare bones
>>> for the purposes of this mail.
>>>
>>>
>>>
>>> I have an Erlang 18.1 application that uses ETS to store an Erlang map
>>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to
>>> be 1MB. Upon the necessary activity trigger the application spawns about 25
>>> short-lived processes to perform the main work of the application. This
>>> activity trigger is fired roughly 9 times a second under normal operating
>>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to
>>> read from the map.
>>>
>>>
>>>
>>> What I've found is that the above implementation has a CPU profile that
>>> is quite "expensive" - each of the CPU cores (40 total comprised of 2
>>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The
>>> machine in question also has 32GB RAM of which about 9GB is used at peak.
>>> There is no swap usage whatsoever. Examination shows that copy_shallow is
>>> performing the most work.
>>>
>>>
>>>
>>> After changing the implementation so that the 25 spawned processes no
>>> longer read from the ETS table to retrieve the map structure and, instead
>>> the map is passed to the processes on spawn, the CPU usage on the server is
>>> considerably lower.
>>>
>>>
>>>
>>> Can anyone offer advice as to why I'm seeing the differing CPU profiles?
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160316/a612d18b/attachment.htm>