[erlang-questions] Re: Conceptual questions on key-value databases for RDBMs users

[Joe Armstrong]

I agree - I'd go further. For most of what I want to do the weaker the
type the better - even dynamic types are too structured. A lot of what
I want can be done with unstructured text and full-text indexing
(which has even less structure that a weak type)

I find that in all the applications I've programmed so far I want the
following (In priority order)

0) Easy of setup
    I'm lazy - but I don't want to struggle through hour long sessions of
    Googling for bug-fixes to get stuff working.
1) Key - Value lookup/retrieval
    What I put in I can get out - the key is simple the Value can be
very complex
2) Full text searches
    I want to search strings for free text
3) Encapsulation
    I like all the database in a single file - so I can send it in a
message move it around etc.
4) Replication
    I want stuff to be stored forever
5) Scalability
    I want it to scale


Most apps I can do with 1 and 2 - I really like 3 for distributing the
result of something.
Ets+Dets is pretty ok for 1+3+0. Mnesia for 4. For 5 I guess
riak/couchDB are good
but I haven't felt the need yet.

I've actually used poor-man's replication, this is pure KISS stuff.
Put the database in a dets file
and replicate using dropbox.

I feel that 2) is the big thing that missing at the moment - dets/ets
with in-built full text
indexing would be *wonderful* - I've made a full-text indexing engine
(See https://github.com/joearms/elib1/tree/master/supported/indexer/)
But it's not integrated with ets/dets.

Does anybody know of any pure-erlang add-ons to ets/dets that do
full-text indexing?
the tricky but seems to be word extraction and not
making/sorting/building the index
also building an incremental index might be difficult (I haven't done this).

I have never seen a "one-size-fits-all" database - and I haven't even mentioned
security availability ...

I just don't believe that one paradigm can suit all needs - what I
want is a toolkit of
pluggable components that do single things and do them well - ets and
dets are two such
components - missing components are the full-text-indexer, the
replication manager, ...

A number of components that can be glued together is what I want not a
pre-packaged solution.

Right now what I want is dets/ets with full-text indexing and search ...

/Joe





On Sun, Nov 7, 2010 at 4:06 PM, Steve Davis
<steven.charles.davis@REDACTED> wrote:
> On Nov 7, 6:09 am, Rudolph van Graan <r...@REDACTED> wrote:
>> Invariance is a first order concept in any RDBMS system and essential for many types of systems. However, in my experience, you can achieve application-level invariance in other ways. But it requires substantially more work to approximate first order invariance using KV-stores.  It is sometimes simpler to have your data properly normalised I think.
>
> "not an expert" caveat applies to me also...
>
> It appears to me that this discussion is another expression of the
> 'strong vs weak/dynamic vs static type' discussion.
>
> Some observations/connections that seem relevant to me:
> - The underlying nature of SQL DBs is revealed by the fact that
> queries by anything except the primary key usually need additional
> work (sequences/indexes) to be respectably efficient.
> - C.J.Date uses the term tuple to refer to a "row in a table" (cf also
> Mnesia).
> - Date also notes that "objects" can always be mapped directly on to a
> relational model.
> - Mnesia is interesting in that it encourages a table to be confined
> to a single record type, and allows indexing of record tuple members
> (cf column indexing in SQL databases).
> - The process of normalization suggests very strong typing of the
> data.
>
> ...it makes me suspect that an imperative and strongly-typed language
> paradigm has been a very strong motivator in the evolution of SQL
> databases; and perhaps the popularity of NoSQL/NotSQL is an expression/
> outcome of the rise of recent trends in programming language uptake.
>
> /s
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>