[erlang-questions] data managment layer on top of K-V stores

Joe Armstrong erlang@REDACTED
Wed Feb 29 14:40:07 CET 2012


Has anybody made (in Erlang) a database managing layer that sits on top of
a Key-Value store.

What I imagine is the following:

    1) at the bottom there is a K-V store  with a primitive set of operations
        get(key) put(key,Val) foldkeys(F, Acc) foldKV(F, Acc)

    2) At the top level - I have typed record (say)

      -record(person, {name :: binary(), arg:: integer, ...})

      I'd like the say

            db:store(#person{...})

      To store a new tuple in the database - this would involve a few
locked KV operations on the
       underlying KV store. I'd have to fetch a schema, type check the
arguments, made a new index and so

       I'd like to also search the database

          db:search([{record,person},{name,{matchesRegExp,...}])

       and so on

   There seem to be a lot of alternative for the bottom layer - but
what about the glue between the bottom layer
    and the top level - has anybody done any work on this? (I guess a
lot of mnesia does this - since the bottom layers
    are just ets and dets) but I'd like to play with other bottom layers.

   I  also need to lock the KV store for short times - instead of the
regular get and put I feel I'd like a
function

         withKeys( [ Key ], fun doit/1)

        which looks up Key1, Key2, .. in [ Keys] and assumes this
returns a list L = [{K1, {yes, V1} }, {K2, no} etc.
        it then calls doit(L) - doit(L) must return {Val, [{K,V}]}

        Val is the return value and the list [{K,V}] is injected back
into the store.

         This is a simple form of atomic transaction which assumes
that all the write are performed only if doit succeeds
         otherwise the store is not changed.

     Pure K-V stores seem to need a little bit of extra glue on top -
some kind of atomic transactions and type checking and
     schema control - mnesia does all of this - but the glue seems to
be inseparable so I can't use it with a different backend.


   /Joe



More information about the erlang-questions mailing list