[erlang-questions] where it's the best way to store a very big term object shared between processes

[Antonio SJ Musumeci]

If it almost never or never changes after the first computation then the
mochiglobal way will probably work fine. I had thrown together a simple
library to expand on the idea[0]. I've used it to serve assets in a cowboy
server that was escriptized. Fast and easy to distribute/deploy.

[0] http://github.com/trapexit/wandb

On Thu, Oct 22, 2015 at 10:10 AM, Sean Cribbs <seancribbs@REDACTED> wrote:

> I'd echo Richard's message. This is exactly what the unfortunately-named
> 'mochiglobal' module does. If you have a large, low-write, high-read
> data-structure, it's perfect. Keep in mind that compiling the module may
> take a lot longer than copying a large datastructure between processes, and
> so be REALLY REALLY sure this is what you want first.
>
> In the early days of Riak we used mochiglobal to store the ring, because
> then many client processes could access it at once without going through a
> gen_server. However, over time we realized the churn was too high for the
> code server and switched to something else; I think it ended up being a
> gen_server handling updates but putting it in a public-read ets table.
>
> On Thu, Oct 22, 2015 at 8:18 AM, Richard Carlsson <
> carlsson.richard@REDACTED> wrote:
>
>> If the data is constant once the table is built, and the entries are big
>> so you don't want to copy them out from an ets table or even avoid passing
>> them between processes (if you want to distribute the table to a large
>> number of processes), you could generate a module, e.g. using Merl, with a
>> lookup function that does what the trie lookup would do. It might take some
>> effort but the speedup could be huge.
>>
>>
>>         /Richard
>>
>> On Thu, Oct 22, 2015 at 1:56 PM, Jesper Louis Andersen <
>> jesper.louis.andersen@REDACTED> wrote:
>>
>>>
>>> On Thu, Oct 22, 2015 at 1:19 PM, Adam Krupička <akrupicka@REDACTED>
>>> wrote:
>>>
>>>> The gen_server approach would be more efficient; most efficient would be
>>>> storing a copy of the trie in each process (but also most memory
>>>> wasteful).
>>>>
>>>
>>> Alternatively, level compress[0] the first few levels into N processes.
>>>
>>> [0] Technically you then get a concurrent level-compressed trie.
>>>
>>>
>>> --
>>> J.
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20151022/48491c22/attachment.htm>