[ANN] ram, an in-memory distributed KV store

Jesse Gumm gumm@REDACTED
Tue Dec 21 16:40:26 CET 2021

Very cool.  Also, interesting timing.

Just about a week ago, I started work on a distributed KV store for session
management (mostly for Nitrogen, but no reason it can't be general purpose).


The distributed component is not yet functional (that's today's focus), but
the point of this is to be:

* fully replicated on all nodes
* self healing
* expiration based on last session access time
* data persistence (because I don't like losing session data on a VM
restart for a single-node webapp)
* easy adding or dropping from the cluster

So it's definitely not a general purpose KV by any stretch, but it's
interesting the overlap here at the same time (our initial commits on these
projects were less than a day apart).

I will definitely be keeping an eye on ram.


On Tue, Dec 21, 2021 at 8:45 AM Marc Worrell <marc@REDACTED> wrote:

> We have https://github.com/zotonic/depcache
> This is a NON-distributed caching system which:
>  - tracks dependencies between keys
>  - cache expiration
>  - local in process memoization of lookups
>  - automatic cleanup with a max memory usage (soft limit)
> I have been pondering to make this distributed for a long time….
> Cheers,
> Marc
> On 21 Dec 2021, at 14:57, Roberto Ostinelli <ostinelli@REDACTED> wrote:
> Let’s write a database! Well not really, but I think it’s a little sad
> that there doesn’t seem to be a simple in-memory distributed KV database in
> Erlang. Many times all I need is a consistent distributed ETS table.
> The two main ones I normally consider are:
>    - *Riak* which is great, it handles loads of data and is based on
>    DHTs. This means that when there are cluster changes there is a need for
>    redistribution of data and the process needs to be properly managed, with
>    handoffs and so on. It is really great but it’s eventually consistent and
>    on many occasions it may be overkill when all I’m looking for is a simple
>    in-memory ACI(not D) KV solution which can have 100% of its data replicated
>    on every node.
>    - *mnesia* which could be it, but unfortunately requires special
>    attention when initializing tables and making them distributed (which is
>    tricky), handles net splits very badly, needs hacks to resolve conflicts,
>    and does not really support dynamic clusters (additions can be kind of ok,
>    but for instance you can’t remove nodes unless you stop the app).
>    - …other solutions? In general people end up using Foundation DB or
>    REDIS (which has master-slave replication), so external from the beam.
>    Pity, no?
> So… :) Well I don’t plan to write a database (since ETS is *awesome*),
> rather distributing it in a cluster. I’d simply want a distributed ETS
> solution after all!
> I’ve already started the work and released a version 0.1.0 or ram:
> https://github.com/ostinelli/ram
> Docs are here:
> https://hexdocs.pm/ram
> Please note this is a very early stage. It started as an experiment and it
> might remain one. So feedback is welcome to decide its future!
> Best,
> r.

Jesse Gumm
Owner, Sigma Star Systems
414.940.4866 || sigma-star.com || @jessegumm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20211221/111dfed3/attachment.htm>

More information about the erlang-questions mailing list