Segmented Cache strategy

Nelson Vides nelson.vides@REDACTED
Thu Oct 7 16:23:03 CEST 2021

I’ve created a new Erlang cache library, from the lessons and needs we had in MongooseIM, called segmented_cache, code available at <> and hex package here <>, and I’m looking for opinions, thoughts, kudos, or maybe even contributions :)

1. How does it work: it uses a coarse-grained strategy, that is, it keeps a set of ETS tables that are periodically rotated, and on rotation, the last table is cleared.

2. What is different from all other cache libraries I saw:
* Time-to-live: entries are inserted in buckets (ets tables) that are entirely dropped when their ttl is triggered, making ttls very coarsed-grained, and book-keeping extremely simple and performant.
* Extensible values: we can configure a callback to merge old and new values upon key insert conflict, like, if values are maps, we can use maps:merge/2.
* Concurrency: this implementation is specially tailored for massively concurrent cache lookups and inserts: many caches in Erlang read from an ets table with read_concurrency but serialise inserts through a gen_server, which, in high insert bursts, becomes a bottleneck, not considering duplicates inserts that infinitely many workers might request. Other caches serialise read and writes through a gen_server that keeps the data in a map in its state: this is not only a bottleneck but a source of garbage, as the map is copied on every mutation. Here instead, the cache uses a –bunch of– ets table with read_concurrency, write_concurrency, and decentralized_counters, to maximise concurrent throughput of all read and writes. It also stores references to the buckets, the index, and the merge strategy, using persistent_term, to maximise performance.
* Instrumentation: lookups raise telemetry events with metadata about cache hits or misses, and the duration of the lookup operation, aiding at following the RED method.

Hope somebody finds it useful, and I get some feedback on the idea :)
upcoming conferences: *

ElixirConf EU: 
<> 8-10 September
Beam SF: <> 4-5 November

Erlang Solutions cares about your data and privacy;
please find 
all details about the basis for communicating with you and the way we
process your data in our Privacy Policy 
<>. You can
update your 
email preferences or opt-out from receiving Marketing emails here 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the erlang-questions mailing list