persistent_term to replace ETS for caching

Rickard Green rickard@REDACTED
Mon Dec 7 00:56:26 CET 2020

On Sun, Dec 6, 2020 at 5:58 PM Frank Muller <frank.muller.erl@REDACTED>

> More info on the use-case:
> The entries in the cache are created and deleted by a single unique
>  process in the system. They can stay inside the cache for hours, days or
> even weeks before being erased.
> All other processes only read from cache.
> There is no update to entries. Once they are created they stay unchanged
> until they get deleted.
> Does anyone know if the VM can efficiently create and handle millions of
> persistent_term(s)?
I don't think you want 5.3 million keys in persistent term. The hash table
used for lookup of the keys is recreated at every update.


> /Frank
> Sun 6  Dec 2020 17:40, Bob Ippolito <bob@REDACTED>  wrote :
>> I’m not familiar with the implementation details but my understanding
>> from the docs is that erase/1 and put/2 (only when updating an existing
>> term) cause a global GC. What processes do with those terms is only
>> relevant in that if they have references to multi-word persistent terms
>> that are being updated or erased then the global GC pause will take longer.
>> On Sun, Dec 6, 2020 at 06:29 Richard O'Keefe <raoknz@REDACTED> wrote:
>>> Thanks for that advice about persistent_term.
>>> From the documentation,
>>> <quote>
>>> When a persistent term is updated or deleted, a global garbage
>>> collection pass is run to scan all processes for the deleted term, and to
>>> copy it into each process that still uses it.
>>> </quote>
>>> Do I understand correctly that if three processes refer
>>> to a persistent-term, and one of them deletes it, it
>>> will be copied into both of the other processes, thus
>>> INCREASING the amount of memory used?
>>> This is sufficiently counter-intuitive that I am not
>>> sure I would dare to use this feature.
>>> Do I further understand correctly that *adding* a new
>>> persistent-term does NOT force garbage collection?
>>> On Mon, 7 Dec 2020 at 00:49, Nalin Ranjan <ranjanified@REDACTED> wrote:
>>>> It has the potential to trigger Global GC, and can affect
>>>> responsiveness as per the docs.
>>>> Regards
>>>> Nalin Ranjan
>>>> On Sun, Dec 6, 2020 at 5:16 AM Frank Muller <frank.muller.erl@REDACTED>
>>>> wrote:
>>>>> Hi guys,
>>>>> At work, we cache about 5.3 million entries in ETS. The system works
>>>>> perfectly, no issue so far (many years).
>>>>> During a brainstorming session, a colleague suggested to switch to
>>>>> persistent_term instead to avoid ETS term copying.
>>>>> Pretty simple: we check if the Key exists in persistent_term. If yes,
>>>>> we are done. If not, we get it from ETS, move it to persistent_term and
>>>>> send it back to the caller.
>>>>> Question: is there any limitation(s) on persistent_term usage? Stated
>>>>> otherwise, can we create 5.3 million persistent_term <K,V>?
>>>>> Any suggestion/idea/thought is very welcome.
>>>>> /Frank

Rickard Green, Erlang/OTP, Ericsson AB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the erlang-questions mailing list