[erlang-questions] ETS and CPU

[Alex Howle]

The map is not being sent as a message. It is passed in to the spawned
processes by being in scope of the spawned function.

Pseudocode:

A=bigmap,
spawn(fun()-> do_something(A) end).
On 15 Mar 2016 13:43, "Alex Howle" <itshowlertime@REDACTED> wrote:

> The map is not being sent as a message. It is passed in to the spawned
> processes by being in scope of the spawned function.
>
> Pseudocode:
>
> A=bigmap,
> spawn(fun()-> do_something(A) end).
> On 15 Mar 2016 11:32, "Sverker Eriksson" <sverker.eriksson@REDACTED>
> wrote:
>
>> Each successful ets:lookup call is a copy operation of the entire term
>> from ETS to the process heap.
>>
>> If you are comparing ets:lookup of big map
>> to sending big map in message then I would expect
>> ets:lookup to win, as copy_shallow (used by ets:lookup)
>> is optimized to be faster than copy_struct (used by send).
>>
>>
>> /Sverker, Erlang/OTP
>>
>>
>> On 03/15/2016 09:52 AM, Alex Howle wrote:
>>
>> I've been experiencing an issue and was wondering if anyone else has any
>> experience in this area. I've stripped back the problem to its bare bones
>> for the purposes of this mail.
>>
>>
>>
>> I have an Erlang 18.1 application that uses ETS to store an Erlang map
>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to
>> be 1MB. Upon the necessary activity trigger the application spawns about 25
>> short-lived processes to perform the main work of the application. This
>> activity trigger is fired roughly 9 times a second under normal operating
>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to
>> read from the map.
>>
>>
>>
>> What I've found is that the above implementation has a CPU profile that
>> is quite "expensive" - each of the CPU cores (40 total comprised of 2
>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The
>> machine in question also has 32GB RAM of which about 9GB is used at peak.
>> There is no swap usage whatsoever. Examination shows that copy_shallow is
>> performing the most work.
>>
>>
>>
>> After changing the implementation so that the 25 spawned processes no
>> longer read from the ETS table to retrieve the map structure and, instead
>> the map is passed to the processes on spawn, the CPU usage on the server is
>> considerably lower.
>>
>>
>>
>> Can anyone offer advice as to why I'm seeing the differing CPU profiles?
>>
>>
>> _______________________________________________
>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160315/252bb70d/attachment.htm>