[erlang-questions] where it's the best way to store a very big term object shared between processes

[Jesper Louis Andersen]

On Mon, Nov 23, 2015 at 12:06 PM, Benoit Chesneau <bchesneau@REDACTED>
wrote:

>
> I am not sure to understand the solution. Do you mean copying the map to
> different processes? What would be a good default number of processes then
> o start to copy the data? (/me is thinking N = number of cpus but...)
>
>
maps() has no sharing, so in that case you need a single process to contain
all of the data. Or you would have to split the data among several
processes by level compressing the first levels of the radix tree. But this
assumes you have roughly uniform distribution in your lookups.


> Related to that I am wondering what `read_concurrency` means internally?
>

Mutexes around ETS are RX-mutexes. That is, they support either many
readers or one (eXclusive) writer. But there are several ways to construct
such an RX-mutex.

In particular, you can make sure that if two or more cores want to read,
they don't need to synchronize caches and be coherent. Such a
"reader-optimized" mutex is selected when you enable read_concurrency. It
makes writes more expensive in the sense you have to grab locks across all
cores, but it makes the common read much faster.

I think Rickard Green and/or Sverker Eriksson and/or Björn-Egil Dahlberg
knows more about these details if you are interested.

-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20151123/2a0e32d3/attachment.htm>