[erlang-questions] where it's the best way to store a very big term object shared between processes

[Sean Cribbs]

I'd echo Richard's message. This is exactly what the unfortunately-named
'mochiglobal' module does. If you have a large, low-write, high-read
data-structure, it's perfect. Keep in mind that compiling the module may
take a lot longer than copying a large datastructure between processes, and
so be REALLY REALLY sure this is what you want first.

In the early days of Riak we used mochiglobal to store the ring, because
then many client processes could access it at once without going through a
gen_server. However, over time we realized the churn was too high for the
code server and switched to something else; I think it ended up being a
gen_server handling updates but putting it in a public-read ets table.

On Thu, Oct 22, 2015 at 8:18 AM, Richard Carlsson <
carlsson.richard@REDACTED> wrote:

> If the data is constant once the table is built, and the entries are big
> so you don't want to copy them out from an ets table or even avoid passing
> them between processes (if you want to distribute the table to a large
> number of processes), you could generate a module, e.g. using Merl, with a
> lookup function that does what the trie lookup would do. It might take some
> effort but the speedup could be huge.
>
>
>         /Richard
>
> On Thu, Oct 22, 2015 at 1:56 PM, Jesper Louis Andersen <
> jesper.louis.andersen@REDACTED> wrote:
>
>>
>> On Thu, Oct 22, 2015 at 1:19 PM, Adam Krupička <akrupicka@REDACTED>
>> wrote:
>>
>>> The gen_server approach would be more efficient; most efficient would be
>>> storing a copy of the trie in each process (but also most memory
>>> wasteful).
>>>
>>
>> Alternatively, level compress[0] the first few levels into N processes.
>>
>> [0] Technically you then get a concurrent level-compressed trie.
>>
>>
>> --
>> J.
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20151022/803d474b/attachment.htm>