[erlang-questions] Storing Map into DB

[Jesper Louis Andersen]

On Thu, Mar 23, 2017 at 10:51 PM Danniel Condez <ducondez@REDACTED> wrote:

> Hi,
>
> Any recommendation for persisting a Map into an unstructured
> db/filesystem. The goal is to store and fetch it as is, so that both erlang
> backend and FrontEnd (GraphQL) can easily fetch it.
>
>
Our GraphQL system uses Postgres as a database and utilizes jsonb columns
to store map data inside the database. The tradeoff is that your map keys
are binary(), but the advantage is that you have the ability to add indexes
over your jsonb columsn for quicker access to report-style queries in your
data set.

We essentially took Facebook's TAO-paper (Types, Associations & Objects)
and implemented it on top of Postgres rather than MySQL. We can then use
the TAO-API in order to satisfy queries in the GraphQL system. Preliminary
analysis and benchmarking shows that this works exceptionally well. Most
queries are satisfied by index lookups from memory and is in the sub 3ms
range, even for larger blobs of data.

Our current primary gotcha is that you can't define

-spec m() :: #{ <<"foo">> := integer(), <<"bar">> := integer() }.

for the dialyzer. So we can't make the dialyzer efficient at debugging
right now. However, the keys are static keys, so we could probably convert
them into atom()'s without too much hassle. They are limited in what they
can be. This requires some future rework of our GraphQL layer as well
though.

(For those interested in the GraphQL stuff: talk and release is currently
slated for EUC 2017).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20170324/d1cbf0f3/attachment.htm>