[erlang-questions] ETS and CPU

[Hynek Vychodil]

I'm sorry, there is a typo. It should be spawn_opt(fun()-> do_something()
end, [{min_heap_size, Size}]).

On Tue, Mar 15, 2016 at 7:13 PM, Hynek Vychodil <vychodil.hynek@REDACTED>
wrote:

> If you start it in this way, your map is copied from a heap of the parent
> process to a heap of the new process. The difference could be starting size
> of a heap. Sou you could try to start the process as spawn_op(fun()->
> do_something(A) end, {}min_heap_size, Size}).
> With Size around 2 * erts_debug:flat_size(Map).
>
> On Tue, Mar 15, 2016 at 4:07 PM, Alex Howle <itshowlertime@REDACTED>
> wrote:
>
>> The map is not being sent as a message. It is passed in to the spawned
>> processes by being in scope of the spawned function.
>>
>> Pseudocode:
>>
>> A=bigmap,
>> spawn(fun()-> do_something(A) end).
>> On 15 Mar 2016 13:43, "Alex Howle" <itshowlertime@REDACTED> wrote:
>>
>>> The map is not being sent as a message. It is passed in to the spawned
>>> processes by being in scope of the spawned function.
>>>
>>> Pseudocode:
>>>
>>> A=bigmap,
>>> spawn(fun()-> do_something(A) end).
>>> On 15 Mar 2016 11:32, "Sverker Eriksson" <sverker.eriksson@REDACTED>
>>> wrote:
>>>
>>>> Each successful ets:lookup call is a copy operation of the entire term
>>>> from ETS to the process heap.
>>>>
>>>> If you are comparing ets:lookup of big map
>>>> to sending big map in message then I would expect
>>>> ets:lookup to win, as copy_shallow (used by ets:lookup)
>>>> is optimized to be faster than copy_struct (used by send).
>>>>
>>>>
>>>> /Sverker, Erlang/OTP
>>>>
>>>>
>>>> On 03/15/2016 09:52 AM, Alex Howle wrote:
>>>>
>>>> I've been experiencing an issue and was wondering if anyone else has
>>>> any experience in this area. I've stripped back the problem to its bare
>>>> bones for the purposes of this mail.
>>>>
>>>>
>>>>
>>>> I have an Erlang 18.1 application that uses ETS to store an Erlang map
>>>> structure. Using erts_debug:flat_size/1 I can approximate the map's size to
>>>> be 1MB. Upon the necessary activity trigger the application spawns about 25
>>>> short-lived processes to perform the main work of the application. This
>>>> activity trigger is fired roughly 9 times a second under normal operating
>>>> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to
>>>> read from the map.
>>>>
>>>>
>>>>
>>>> What I've found is that the above implementation has a CPU profile that
>>>> is quite "expensive" - each of the CPU cores (40 total comprised of 2
>>>> Processors with 10 hyperthreaded cores) frequently runs at 100%. The
>>>> machine in question also has 32GB RAM of which about 9GB is used at peak.
>>>> There is no swap usage whatsoever. Examination shows that copy_shallow is
>>>> performing the most work.
>>>>
>>>>
>>>>
>>>> After changing the implementation so that the 25 spawned processes no
>>>> longer read from the ETS table to retrieve the map structure and, instead
>>>> the map is passed to the processes on spawn, the CPU usage on the server is
>>>> considerably lower.
>>>>
>>>>
>>>>
>>>> Can anyone offer advice as to why I'm seeing the differing CPU profiles?
>>>>
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>>
>>>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160315/303acbf4/attachment.htm>