Segmented Cache strategy

[Nelson Vides]

I’ve created a new Erlang cache library, from the lessons and needs we had in MongooseIM, called segmented_cache, code available at https://github.com/esl/segmented_cache <https://github.com/esl/segmented_cache> and hex package here https://hex.pm/packages/segmented_cache <https://hex.pm/packages/segmented_cache>, and I’m looking for opinions, thoughts, kudos, or maybe even contributions :)

1. How does it work: it uses a coarse-grained strategy, that is, it keeps a set of ETS tables that are periodically rotated, and on rotation, the last table is cleared.

2. What is different from all other cache libraries I saw:
* Time-to-live: entries are inserted in buckets (ets tables) that are entirely dropped when their ttl is triggered, making ttls very coarsed-grained, and book-keeping extremely simple and performant.
* Extensible values: we can configure a callback to merge old and new values upon key insert conflict, like, if values are maps, we can use maps:merge/2.
* Concurrency: this implementation is specially tailored for massively concurrent cache lookups and inserts: many caches in Erlang read from an ets table with read_concurrency but serialise inserts through a gen_server, which, in high insert bursts, becomes a bottleneck, not considering duplicates inserts that infinitely many workers might request. Other caches serialise read and writes through a gen_server that keeps the data in a map in its state: this is not only a bottleneck but a source of garbage, as the map is copied on every mutation. Here instead, the cache uses a –bunch of– ets table with read_concurrency, write_concurrency, and decentralized_counters, to maximise concurrent throughput of all read and writes. It also stores references to the buckets, the index, and the merge strategy, using persistent_term, to maximise performance.
* Instrumentation: lookups raise telemetry events with metadata about cache hits or misses, and the duration of the lookup operation, aiding at following the RED method.

Hope somebody finds it useful, and I get some feedback on the idea :)
Nelson.
-- 
*
*
*Our
upcoming conferences: *

ElixirConf EU: 
<https://www2.elixirconf.eu/elixir-conf-2021/es> 8-10 September
2021
Code 
Beam SF: <https://www2.codesync.global/code-beam-sf-2021/es> 4-5 November
2021







Erlang Solutions cares about your data and privacy;
please find 
all details about the basis for communicating with you and the way we
process your data in our Privacy Policy 
<https://www.erlang-solutions.com/privacy-policy.html>. You can
update your 
email preferences or opt-out from receiving Marketing emails here 
<https://www2.erlang-solutions.com/email-preference?epc_hash=JtO6C7Q2rJwCdZxBx3Ad8jI2D4TJum7XcUWcgfjZ8YY>.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20211007/dc918628/attachment.htm>