[erlang-questions] data managment layer on top of K-V stores

[Tim Watson]

Sounds like yet another use case for 'protocols' or whatever. This is one of so many examples where we want to interact with many implementations of something via a common API. I think there is some stuff in BOSSDB that you might find does part of what you want, at least for riak and tokyo tyrant. Maybe worth a peek.

On 29 Feb 2012, at 13:40, Joe Armstrong wrote:

> Has anybody made (in Erlang) a database managing layer that sits on top of
> a Key-Value store.
> 
> What I imagine is the following:
> 
>    1) at the bottom there is a K-V store  with a primitive set of operations
>        get(key) put(key,Val) foldkeys(F, Acc) foldKV(F, Acc)
> 
>    2) At the top level - I have typed record (say)
> 
>      -record(person, {name :: binary(), arg:: integer, ...})
> 
>      I'd like the say
> 
>            db:store(#person{...})
> 
>      To store a new tuple in the database - this would involve a few
> locked KV operations on the
>       underlying KV store. I'd have to fetch a schema, type check the
> arguments, made a new index and so
> 
>       I'd like to also search the database
> 
>          db:search([{record,person},{name,{matchesRegExp,...}])
> 
>       and so on
> 
>   There seem to be a lot of alternative for the bottom layer - but
> what about the glue between the bottom layer
>    and the top level - has anybody done any work on this? (I guess a
> lot of mnesia does this - since the bottom layers
>    are just ets and dets) but I'd like to play with other bottom layers.
> 
>   I  also need to lock the KV store for short times - instead of the
> regular get and put I feel I'd like a
> function
> 
>         withKeys( [ Key ], fun doit/1)
> 
>        which looks up Key1, Key2, .. in [ Keys] and assumes this
> returns a list L = [{K1, {yes, V1} }, {K2, no} etc.
>        it then calls doit(L) - doit(L) must return {Val, [{K,V}]}
> 
>        Val is the return value and the list [{K,V}] is injected back
> into the store.
> 
>         This is a simple form of atomic transaction which assumes
> that all the write are performed only if doit succeeds
>         otherwise the store is not changed.
> 
>     Pure K-V stores seem to need a little bit of extra glue on top -
> some kind of atomic transactions and type checking and
>     schema control - mnesia does all of this - but the glue seems to
> be inseparable so I can't use it with a different backend.
> 
> 
>   /Joe
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions