[erlang-questions] How to code against an interface rather than an implementation?

[Joe Armstrong]

On Fri, Aug 9, 2013 at 9:41 AM, H.C. v. Stockhausen <hc@REDACTED> wrote:

> Hello,
>
> I need a DB backend for my application but I'd like to be able to swap
> it out for different DBs if I choose so later on.
>
> I would like to code against an interface and tell the application
> what specific backend to use through config rather than code changes.
>
> Is there a pattern for doing that? Hot code upgrades and multi-node
> are not a real concern at this time - mostly, since I have no
> experience yet with either - however if doing it right means taking
> that into account too I'd also like to learn more about that.
>
> I am thinking of defining a custom behaviour ("my_crud" perhaps), then
> to implement it for various DBs and to also let a config driven
> adapter implement it that I then use to throughout the code to talk to
> the DB layer.
>
> For example, using Mnesia and AWS DynamoDB:
>
> - my_crud.erl (behaviour)
> - my_db_mnesia.erl (implements behaviour)
> - my_db_dynamo.erl (implements behaviour)
> - my_db.erl (configurable adapter that also implements behaviour)
> - my.config
>
> my_db:insert(Key, Value).
>
> Is that a reasonable approach that makes proper use of Erlang and
> behaviours or is this just not how one should do it?
>

Yes - this is fine

But ...

Firstly, this might be overkill. One of the real reasons for the import
declaration was to be able to
flip the back-end implementation while providing a common interface library:

So we'd write:

    -module(xxx).
    -import(my_mnesia_backend, [insert/2, lookup/1, ...])

    foo(X) ->
          insert(X, ...)

    and so on.

    To change the backend you just change the name of the module in the
import declaration

    That was the idea - but in practice very few libraries are written with
compatible back-ends

The second problem with doing this is that it turns out to be extremely
difficult to make a common
front-end - this is essentially the least common denominator of the all the
services offered by the
individual databases.

There are often very good reasons why databases are different - they offer
different services.

I recently considered making a common front-end for dets and basho's
bitcask - this seems reasonable
both are disk based key-value crash tolerant stores, so this seemed like a
good idea, I'd be able to use one store and then flip to the other if
circumstances changed.

The problem is that dets and bitcask are rather different - they *have* to
be, otherwise why would the basho guys write bitcask. ets and dets play
nicely together (ets:from_dets :-) - ets keys and values are terms. bitcask
keys and values are binaries. Which will the common interface have? - if
you choose (say) terms
you'll bias the interface in favour of dets.

So now we have a problem - two interfaces can be *functionally* equivalent
(ie offer the same interface in terms of types) but the non-functional
properties of the interface might be completely different.

For example database1 might be optimised for fast reads/slow writes.
Database2 might be optimised for fast writes.

It seems to me that the non-functional parts of a system are the bits that
are difficult to get right -
making things (fault- tolerant, scalable, upgradable, ... etc.) is far more
difficult than getting types
and interfaces right.

There are two approaches to making a system with a database:

1) Choose the database first - make your mind up
2) Implement everything

or

1) Make an abstraction layer
2) Write application and interface layers
3) configure and decide on database later

The hope is that by choosing the second approach, you save work since if
you chose the wrong
database you can just change the backend later.

The problem here is that the non-functional behaviour of the backend you
choose causes feature
creep in the front end. If the backend has fast reads and slow writes, the
front-end software will endup having a design that reflects this.

The question you now have to ask is the generalisation and extra interface
libraries worth the extra
effort?

I often start by trying to generalise dissimilar things and make them
similar - but get to a point
where the generalisation and interfacing becomes the main problem and not
solving the original problem.

That was a bit long and rambling ...

So the answer is "Yes you can do this, and it *might* be a good idea" to
see if it *is* a good idea, you'll
have to implement things three times (with two data bases) (ie solution
with database A alone and no abstraction layer) (database B with no
abstraction layer) (A and B with a common abstraction layer)

Often the abstraction layer becomes at least as complex as the thing being
abstracted- at which point
you wish you'd chosen a simpler career like being a brain surgeon or
astronaut.

Cheers

/Joe


> Thank you for any help & best regards,
> Hans
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130809/b4243610/attachment.htm>