[erlang-questions] 1000+ users; 30+ data tables/user

[Garrett Smith]

Hi Lloyd,

Sorry for the late reply here - I was interested in this thread last
week when you sent it but was in sleep deprived conference mode and
never got back to it.

On Tue, Nov 4, 2014 at 8:49 PM, Lloyd R. Prentice <lloyd@REDACTED> wrote:
> Hello,
>
> This is a naive question reflecting my inexperience with databases.
>
> I'm planning to offer my users a set of management/planning tools. Each user would be storing/retrieving user-specific data involving as many as 30 data tables.
>
> --- Data fits well into Erlang records.
> --- We're not talking huge volumes of data per user.
> --- Nor do I expect much data analysis.
> ---  Data integrity and availability are essential.
> --- Users may, however, wish to bundle up their data a some point and migrate to a different system.
>
> I'm attracted to mnesia because of it's it's tight integration with Erlang and it's replication features. I'm also considering riak.
>
> My first thought was that every user would own his/her own database. But this seems to rule out
> mnesia since:
>
> "Mnesia is a truly distributed DBMS and the schema is a system table that is replicated on all nodes in a Mnesia system. The function will fail if a schema is already present on any of the nodes in NodeList."
> http://www.erlang.org/doc/apps/mnesia/Mnesia_chap3.html
>
> An option would be to store data for all users in each of the 30 tables. But is there a better solution altogether?
>
> I'd much appreciate suggestions and guidance from wiser heads.

I think we've all had this problem - a new project and nothing to hold
us back but our own imagination. So the question... which database to
pick. Which indeed? There are like 100 amazing options!

My suggestion here is to stop this line of thinking immediately :)

I would instead plan to throw your early work away. Pick something
that is the fastest and easiest imaginable for you to make progress on
your app. Treat it as a "this will almost certainly not be what I end
up with".

_For me_ this means one of these:

- Hard coded values or config files
- Dets
- SQLite
- MySQL

The point is to keep it as simple as possible and just get stuff working.

When I'm starting on something new, I just don't know enough about
anything to make the right decision - so I deliberately make the right
_wrong_ decision - that is, the decision that will let me move forward
quickly and get to the real problems. I might throw it away later, or
I might keep it. But in any case, I'm sure as hell not going to spend
a lot of time on it. Not until I'm facing real, hard, visible problems
that I can use to inform my next steps.

Garrett