[erlang-questions] ETS and CPU
Wed Mar 16 17:20:24 CET 2016
Well, I would expect copy_shallow (from ETS) to be less CPU intensive
than copy_struct (from process).
However, as indicated by others, ets:lookup on such a big map will probably
trigger a garbage collection on the process, which will lead to
yet another copy of the big map.
The spawn(fun() -> do_something(BigMap) end) on the other hand will
allocate a big enough heap for the process form the start and only do
one copy of the big map.
On 03/16/2016 10:43 AM, Alex Howle wrote:
> Assuming that when you say "win" you mean that ets:lookup should be
> more efficient (and less CPU intensive) then I'm seeing the opposite.
> On 15 Mar 2016 11:32, "Sverker Eriksson"
> < <mailto:>>
> Each successful ets:lookup call is a copy operation of the entire term
> from ETS to the process heap.
> If you are comparing ets:lookup of big map
> to sending big map in message then I would expect
> ets:lookup to win, as copy_shallow (used by ets:lookup)
> is optimized to be faster than copy_struct (used by send).
> /Sverker, Erlang/OTP
> On 03/15/2016 09:52 AM, Alex Howle wrote:
>> I've been experiencing an issue and was wondering if anyone else
>> has any experience in this area. I've stripped back the problem
>> to its bare bones for the purposes of this mail.
>> I have an Erlang 18.1 application that uses ETS to store an
>> Erlang map structure. Using erts_debug:flat_size/1 I can
>> approximate the map's size to be 1MB. Upon the necessary activity
>> trigger the application spawns about 25 short-lived processes to
>> perform the main work of the application. This activity trigger
>> is fired roughly 9 times a second under normal operating
>> conditions. Each of these 25 processes performs 1 x ets:lookup/2
>> calls to read from the map.
>> What I've found is that the above implementation has a CPU
>> profile that is quite "expensive" - each of the CPU cores (40
>> total comprised of 2 Processors with 10 hyperthreaded cores)
>> frequently runs at 100%. The machine in question also has 32GB
>> RAM of which about 9GB is used at peak. There is no swap usage
>> whatsoever. Examination shows that copy_shallow is performing the
>> most work.
>> After changing the implementation so that the 25 spawned
>> processes no longer read from the ETS table to retrieve the map
>> structure and, instead the map is passed to the processes on
>> spawn, the CPU usage on the server is considerably lower.
>> Can anyone offer advice as to why I'm seeing the differing CPU
>> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions