[erlang-questions] ETS and CPU

[Jesper Louis Andersen]

If your map is "flat" and you frequently access individual keys, then you
can avoid copying by restructuring:

If you have {Key, Map} then store [{{Key, K}, V} || {K, V} <-
maps:to_list(Map)] and look up by using {Key, K} where K is the desired map
key. This can break the large map into smaller chunks which may be more
amenable to copying around.

On Tue, Mar 15, 2016 at 12:32 PM, Sverker Eriksson <
sverker.eriksson@REDACTED> wrote:

> Each successful ets:lookup call is a copy operation of the entire term
> from ETS to the process heap.
>
> If you are comparing ets:lookup of big map
> to sending big map in message then I would expect
> ets:lookup to win, as copy_shallow (used by ets:lookup)
> is optimized to be faster than copy_struct (used by send).
>
>
> /Sverker, Erlang/OTP
>
>
>
> On 03/15/2016 09:52 AM, Alex Howle wrote:
>
> I've been experiencing an issue and was wondering if anyone else has any
> experience in this area. I've stripped back the problem to its bare bones
> for the purposes of this mail.
>
>
>
> I have an Erlang 18.1 application that uses ETS to store an Erlang map
> structure. Using erts_debug:flat_size/1 I can approximate the map's size to
> be 1MB. Upon the necessary activity trigger the application spawns about 25
> short-lived processes to perform the main work of the application. This
> activity trigger is fired roughly 9 times a second under normal operating
> conditions. Each of these 25 processes performs 1 x ets:lookup/2 calls to
> read from the map.
>
>
>
> What I've found is that the above implementation has a CPU profile that is
> quite "expensive" - each of the CPU cores (40 total comprised of 2
> Processors with 10 hyperthreaded cores) frequently runs at 100%. The
> machine in question also has 32GB RAM of which about 9GB is used at peak.
> There is no swap usage whatsoever. Examination shows that copy_shallow is
> performing the most work.
>
>
>
> After changing the implementation so that the 25 spawned processes no
> longer read from the ETS table to retrieve the map structure and, instead
> the map is passed to the processes on spawn, the CPU usage on the server is
> considerably lower.
>
>
>
> Can anyone offer advice as to why I'm seeing the differing CPU profiles?
>
>
> _______________________________________________
> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160315/845b8916/attachment.htm>