[erlang-questions] data managment layer on top of K-V stores

[Evan Miller]

Hi Joe,

As mentioned, BossDB might do what you are looking for

https://github.com/evanmiller/boss_db

It supports many databases and many query operators, and just last
week I pushed some code so you can declare each parameter's type and
have it validated.

$ cat puppy.erl
-module(puppy, [Id, Name::string(), BirthDate::datetime()]).

$ erl
1> Puppy = puppy:new(id, "Rex", erlang:now()).
{puppy, id, "Rex", {1330,527688,550808}}.
2> Puppy:name().
"Rex"
3> Puppy:validate_types().
{error, ["Invalid type for birth_date"]} % (expected datetime tuple,
got now/0 timestamp)

Only a few types are supported but I plan to have a more flexible
system in the future.

On Wed, Feb 29, 2012 at 7:40 AM, Joe Armstrong <erlang@REDACTED> wrote:
> Has anybody made (in Erlang) a database managing layer that sits on top of
> a Key-Value store.
>
> What I imagine is the following:
>
>    1) at the bottom there is a K-V store  with a primitive set of operations
>        get(key) put(key,Val) foldkeys(F, Acc) foldKV(F, Acc)
>
>    2) At the top level - I have typed record (say)
>
>      -record(person, {name :: binary(), arg:: integer, ...})
>
>      I'd like the say
>
>            db:store(#person{...})
>
>      To store a new tuple in the database - this would involve a few
> locked KV operations on the
>       underlying KV store. I'd have to fetch a schema, type check the
> arguments, made a new index and so
>
>       I'd like to also search the database
>
>          db:search([{record,person},{name,{matchesRegExp,...}])
>
>       and so on
>
>   There seem to be a lot of alternative for the bottom layer - but
> what about the glue between the bottom layer
>    and the top level - has anybody done any work on this? (I guess a
> lot of mnesia does this - since the bottom layers
>    are just ets and dets) but I'd like to play with other bottom layers.
>
>   I  also need to lock the KV store for short times - instead of the
> regular get and put I feel I'd like a
> function
>
>         withKeys( [ Key ], fun doit/1)
>
>        which looks up Key1, Key2, .. in [ Keys] and assumes this
> returns a list L = [{K1, {yes, V1} }, {K2, no} etc.
>        it then calls doit(L) - doit(L) must return {Val, [{K,V}]}
>
>        Val is the return value and the list [{K,V}] is injected back
> into the store.
>
>         This is a simple form of atomic transaction which assumes
> that all the write are performed only if doit succeeds
>         otherwise the store is not changed.
>
>     Pure K-V stores seem to need a little bit of extra glue on top -
> some kind of atomic transactions and type checking and
>     schema control - mnesia does all of this - but the glue seems to
> be inseparable so I can't use it with a different backend.
>
>
>   /Joe
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions