[erlang-questions] Re: Some tips on Mnesia

[Igor Ribeiro Sucupira]

On Mon, May 24, 2010 at 4:26 PM, Rapsey <rapsey@REDACTED> wrote:
> On Mon, May 24, 2010 at 8:49 PM, Igor Ribeiro Sucupira <igorrs@REDACTED>
> wrote:
>>
>> On Mon, May 24, 2010 at 1:45 AM, Rapsey <rapsey@REDACTED> wrote:
>>
>> I'm just not sure you can efficiently retrieve all the information
>> about a node. Won't the query for this key {'_',NodeName} perform a
>> full table scan?
>
> No it will not actually. I asked this question a on the mailing list not too
> long ago.

In fact, I was not supposing it performs a full table scan on what I
said after that.

But now I'm curious: I ran the following test and it seemed to clearly
be performing a full table scan.

(node1@REDACTED)1> mnesia:create_schema([node()]).
ok
(node1@REDACTED)2> mnesia:start().
ok
(node1@REDACTED)3> mnesia:create_table(test, [{attributes, [k, v]},
{disc_only_copies, [node()]}]).
{atomic,ok}
(node1@REDACTED)4> lists:foreach(fun(N) -> mnesia:dirty_write({test,
{foo, N, N + 1}, N}) end, lists:seq(1, 4000000)).  %% Took a VERY long
time. Try 2 million.  ;-)
ok
(node1@REDACTED)5> timer:tc(mnesia, dirty_select, [test, [{{test,
{foo, '_', 3}, '_'}, [], [ok]}]]).
{4006819,[ok]}
(node1@REDACTED)6> timer:tc(mnesia, dirty_select, [test, [{{test,
{foo, 2, 3}, '_'}, [], [ok]}]]).
{287,[ok]}
(node1@REDACTED)7> timer:tc(mnesia, dirty_select, [test, [{{test,
{foo, '_', 3}, '_'}, [], [ok]}]]).  %% Repetition.
{4109629,[ok]}
(node1@REDACTED)8> timer:tc(mnesia, dirty_select, [test, [{{test,
{foo, 2, 3}, '_'}, [], [ok]}]]).  %% Repetition.
{235,[ok]}


Repeating the procedure with a ram_copies, ordered_set table, I got
349730 microseconds for the matching with the partial key and 23
microseconds for the exact match that returns the same key.

Changing the ram_copies table type to set, I got 288281 microseconds
for the matching with the partial key and 28 microseconds for the
exact match that returns the same key.

So either someone misinformed you or the query must be performed on a
very special way. Is it the second option?

Best regards.
Igor.


> As long as you don't use variable binding ('$1'), it will use the
> index on the first field. {node,{address,NodeName}} is the entire key,
> because I store more than just node information. This way I can use the same
> table for different kinds of data.
>
>>
>> Even if that query was very smart (or you knew in advance what fields
>> you need), it would still need to retrieve several keys, which could
>> even be in different nodes (in the case of a fragmented table).
>> Certainly slower than reading one, longer record (as is the case with
>> a "key-value" table).
>>
>
> Yes your solution might be better for fragmented tables. I don't really use
> mnesia in a distributed environment, so I used something that fits well with
> a single node. This way of saving data is a bit nicer, since you don't have
> to do a proplists:get_value after mnesia:read. But you can still query for
> more than one type of data at the same time.
>
>
>
> Sergej
>
>
>
>
>>
>> So, in short, compared to the "key-value" approach, your solution
>> favors retrieving a single field (only slightly, if the records are
>> smaller than a block/page), but turns it slower to retrieve all the
>> fields of an entity.
>>
>> Best regards.
>> Igor.
>>
>> > This way I can query for that exact information mnesia:(dirty_)read, or
>> > I
>> > can get a list of all node addresses with mnesia:select -
>> > {node,{address,'_'}}, or get a list of all node information with
>> > {node,'_'},
>> > or get all info about a node {node,{'_',NodeName}}.
>> >
>> >
>> > Sergej
>> >
>> > On Sun, May 23, 2010 at 10:46 PM, Igor Ribeiro Sucupira
>> > <igorrs@REDACTED>wrote:
>> >
>> >> Second and third posts:
>> >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-2.html
>> >> http://igorrs.blogspot.com/2010/05/mnesia-one-year-later-part-3.html
>> >>
>> >> On Thu, May 20, 2010 at 12:53 PM, Igor Ribeiro Sucupira
>> >> <igorrs@REDACTED> wrote:
>> >> > For those people who are starting to work with Mnesia, I'm writing a
>> >> > short sequence of blog posts that are intended to point out some
>> >> > relevant facts that don't seem to be well-known and could hurt you in
>> >> > the beginning.
>> >> >
>> >> > I'll try to focus on the information that you can't google.  ;-)  But
>> >> > maybe in the end I'll also summarize the important tips that I
>> >> > learned
>> >> > from other blogs.
>> >> >
>> >> > Here is the first post:
>> >> > http://igorrs.blogspot.com/2010/05/mnesia-one-year-later.html
>> >> >
>> >> > Good luck.
>> >> > Igor.
>> >> >
>> >> > --
>> >> > "The secret of joy in work is contained in one word - excellence. To
>> >> > know how to do something well is to enjoy it." - Pearl S. Buck.
>> >>
>> >> ________________________________________________________________
>> >> erlang-questions (at) erlang.org mailing list.
>> >> See http://www.erlang.org/faq.html
>> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> "The secret of joy in work is contained in one word - excellence. To
>> know how to do something well is to enjoy it." - Pearl S. Buck.
>
>



-- 
"The secret of joy in work is contained in one word - excellence. To
know how to do something well is to enjoy it." - Pearl S. Buck.