<font face="arial" size="2"><p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">Thank you, Joe,</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">You've broadened my horizons and I love the simplification implied.</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">For sake of clarification and deeper understanding:</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">-- Would, then, one directory and n files per user work?</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> One respondent to my question cited possible limits imposed by innodes and atoms. But I</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> see that my work station provides 30138368 inodes and Erlang docs tell me that Erlang</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> provides 1048576 atoms, so I'm not convinced that these limits are a problem.</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">-- Would one directory and n dets tables per user work?</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">-- Can sufficient availability be achieved with conventional backup strategies?</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">Yes, I know the answers are application specific, but are there metrics/rules of thumb/guidelines that can give me back-of-the-envelope answers short of all-out test-and-measurement?</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">Many, many thanks for your insights.</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">Lloyd</p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;"> </p>
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">-----Original Message-----<br />From: "Joe Armstrong" <erlang@gmail.com><br />Sent: Tuesday, November 11, 2014 3:02pm<br />To: "Garrett Smith" <g@rre.tt><br />Cc: "Lloyd R. Prentice" <lloyd@writersglen.com>, "erlang-questions" <erlang-questions@erlang.org><br />Subject: Re: [erlang-questions] 1000+ users; 30+ data tables/user<br /><br /></p>
<div id="SafeStyles1415740288">
<p style="margin:0;padding:0;font-family: arial; font-size: 10pt; word-wrap: break-word;">On Tue, Nov 11, 2014 at 7:41 PM, Garrett Smith <g@rre.tt> wrote:<br />> Hi Lloyd,<br />><br />> Sorry for the late reply here - I was interested in this thread last<br />> week when you sent it but was in sleep deprived conference mode and<br />> never got back to it.<br />><br />> On Tue, Nov 4, 2014 at 8:49 PM, Lloyd R. Prentice <lloyd@writersglen.com> wrote:<br />>> Hello,<br />>><br />>> This is a naive question reflecting my inexperience with databases.<br />>><br />>> I'm planning to offer my users a set of management/planning tools. Each user would be storing/retrieving user-specific data involving as many as 30 data tables.<br />>><br />>> --- Data fits well into Erlang records.<br />>> --- We're not talking huge volumes of data per user.<br />>> --- Nor do I expect much data analysis.<br />>> --- Data integrity and availability are essential.<br />>> --- Users may, however, wish to bundle up their data a some point and migrate to a different system.<br />>><br />>> I'm attracted to mnesia because of it's it's tight integration with Erlang and it's replication features. I'm also considering riak.<br />>><br />>> My first thought was that every user would own his/her own database. But this seems to rule out<br />>> mnesia since:<br />>><br />>> "Mnesia is a truly distributed DBMS and the schema is a system table that is replicated on all nodes in a Mnesia system. The function will fail if a schema is already present on any of the nodes in NodeList."<br />>> http://www.erlang.org/doc/apps/mnesia/Mnesia_chap3.html<br />>><br />>> An option would be to store data for all users in each of the 30 tables. But is there a better solution altogether?<br />>><br />>> I'd much appreciate suggestions and guidance from wiser heads.<br />><br />> I think we've all had this problem - a new project and nothing to hold<br />> us back but our own imagination. So the question... which database to<br />> pick. Which indeed? There are like 100 amazing options!<br />><br />> My suggestion here is to stop this line of thinking immediately :)<br />><br />> I would instead plan to throw your early work away. Pick something<br />> that is the fastest and easiest imaginable for you to make progress on<br />> your app. Treat it as a "this will almost certainly not be what I end<br />> up with".<br />><br />> _For me_ this means one of these:<br />><br />> - Hard coded values or config files<br />> - Dets<br />> - SQLite<br />> - MySQL<br />><br />> The point is to keep it as simple as possible and just get stuff working.<br /><br />Excellent advice.<br /><br />For many systems - I use one file per user. The file contains term_to_binary(X)<br />where X is whatever I feel like to represent the use data.<br /><br />(or you can use text files - then you can run amazing things like grep and find<br />on them :-)<br /><br />The OS caches file access and I can easily analyse/dump the files.<br /><br />I've *never* got to the point where I need to change the file system for<br />a database (but then again I've not built a really big system - and<br />this works fine<br />for several thousand files/users)<br /><br />If and when the design problems are solved you can change representations<br />*if it is necessary* - choosing a database right at the start is<br />"premature optimisation" - if you ever get to this point then the<br />choice of representation<br />should be dictated by measurement and not guesswork.<br /><br />/Joe<br /><br />> When I'm starting on something new, I just don't know enough about<br />> anything to make the right decision - so I deliberately make the right<br />> _wrong_ decision - that is, the decision that will let me move forward<br />> quickly and get to the real problems. I might throw it away later, or<br />> I might keep it. But in any case, I'm sure as hell not going to spend<br />> a lot of time on it. Not until I'm facing real, hard, visible problems<br />> that I can use to inform my next steps.<br />><br />> Garrett<br />> _______________________________________________<br />> erlang-questions mailing list<br />> erlang-questions@erlang.org<br />> http://erlang.org/mailman/listinfo/erlang-questions</p>
</div></font>