mnesia_frag ????

[Ulf Wiger (AL/EAB)]

Sanjaya Vitharana wrote:
>
> "...But I would strongly recommend against using dirty access 
> in combination with fragmented tables."
> 
> "Both read and write in fragmented tables are complex operations."
> 
> If we use the original function without dirty, how it will 
> effect to the efficiency of the huge table something like ~3G 
> (3M rec x 1K).

There is a setup cost when using transactions, which you are
paying anyway if you wrap the whole thing with 
mnesia:activity(transaction, fun() -> ... end, mnesia_frag).
It's in the order of 110 usec on my 1 GHz SunBlade.

The size of the table makes very little difference to
performance, except of course while building or loading
the table. Reads and writes are not affected much at all,
esp. if type is 'set' (a hash table).

Furthermore, there is no difference in this regard between
read() and dirty_read(). The only differences are:
- read() will request a lock on the object before reading,
  while dirty_read() will not. The cost of this operation
  will depend on how many replicas exist on the fragment
  where the object resides. If there's only one copy, the
  operation will amount to a gen_server:call(), a check
  for a table lock, and a check for an object lock, and
  finally, of course, an ets:insert in the lock table.
- read() will perform a lookup in the local transaction
  store, whereas dirty_read() will not.

All in all, a transaction-based read() on my SunBlade,
using a fragmented table with 1000 fragments (no replicas),
takes about 270 us, while a dirty read of the same object
takes about 80 us. Doing the same thing with only 3 fragments
takes about 240 usecs. So, there is a difference, but it's
very small.


> Also, how the number of fragments will effect to the table 
> access?? since the operetions are complex (as you told), can 
> the number of fragments degrade the performance (depending 
> upon the numberof record / table size)? I mean the record 
> access time against the number of fragments (I assume the 
> number of records in the table as constant).

For all practical purposes, I don't think you will notice 
any degradation.


> And also refereing to the
> http://www.erlang.org/ml-archive/erlang-questions/200307/msg00
> 052.html.ORIG
> if we change the access_module configuration parameter to 
> mnesia_frag it says "> without doing any changes to your old 
> code."

This is almost partly true. If e.g. your code uses dirty_write()
and dirty_read() directly, then -- as you have noticed -- the
fragmentation functionality will not work as expected. The 
same thing goes for code that uses the older function
mnesia:transaction(fun() -> ... end).

Also, you will pay a (slight) performance penalty for 
using mnesia_frag on non-fragmented tables. It's about 
a few tens of microseconds per read/write operation.

> Refering to my original question I have used the 
> {disc_copies, NodeList} , so why the mnesia:info() shows only 
> the schema table as disc_copies???

Your original post doesn't show the actual value of 
NodeList. If it's [], then mnesia will by default 
add {ram_copies, [node()]}.

BR,
/Uffe