persistent_term to replace ETS for caching

[Nalin Ranjan]

Sounds like mutation of persistent terms has implications(across the
runtime, affecting everybody, even those who are not involved). For as long
as it is mostly read-only, it should be fine as the docs also notes

*Term lookup (using get/1
<https://erlang.org/doc/man/persistent_term.html#get-1>), is done in
constant time and without taking any locks, and the term is not copied to
the heap (as is the case with terms stored in ETS tables).*

It's the writes that should be cautiously and preferably controlled for the
most optimal usage.

Regards
Nalin Ranjan

On Sun, Dec 6, 2020 at 9:07 PM Zsolt Laky <zsolt.laky@REDACTED> wrote:

> Hi,
>
> You migh take a look at http://blog.erlang.org/persistent_term/
>
> Excellent article by Lukas, answered a lot of questions.
>
> Kind regards,
> Zsolt
>
> Sent from my iPad
>
> On 2020. Dec 6., at 15:30, Richard O'Keefe <raoknz@REDACTED> wrote:
>
> 
> Thanks for that advice about persistent_term.
> From the documentation,
> <quote>
> When a persistent term is updated or deleted, a global garbage collection
> pass is run to scan all processes for the deleted term, and to copy it into
> each process that still uses it.
> </quote>
> Do I understand correctly that if three processes refer
> to a persistent-term, and one of them deletes it, it
> will be copied into both of the other processes, thus
> INCREASING the amount of memory used?
> This is sufficiently counter-intuitive that I am not
> sure I would dare to use this feature.
>
> Do I further understand correctly that *adding* a new
> persistent-term does NOT force garbage collection?
>
>
> On Mon, 7 Dec 2020 at 00:49, Nalin Ranjan <ranjanified@REDACTED> wrote:
>
>> It has the potential to trigger Global GC, and can affect responsiveness
>> as per the docs.
>>
>> https://erlang.org/doc/man/persistent_term.html
>>
>> Regards
>> Nalin Ranjan
>>
>> On Sun, Dec 6, 2020 at 5:16 AM Frank Muller <frank.muller.erl@REDACTED>
>> wrote:
>>
>>> Hi guys,
>>>
>>> At work, we cache about 5.3 million entries in ETS. The system works
>>> perfectly, no issue so far (many years).
>>>
>>> During a brainstorming session, a colleague suggested to switch to
>>> persistent_term instead to avoid ETS term copying.
>>>
>>> Pretty simple: we check if the Key exists in persistent_term. If yes, we
>>> are done. If not, we get it from ETS, move it to persistent_term and send
>>> it back to the caller.
>>>
>>> Question: is there any limitation(s) on persistent_term usage? Stated
>>> otherwise, can we create 5.3 million persistent_term <K,V>?
>>>
>>> Any suggestion/idea/thought is very welcome.
>>>
>>> /Frank
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20201206/a8636703/attachment.htm>