[erlang-questions] dets disk i/o

[Janos Hary]

Dimitry,

Not exactly what you asked, but I measured the write performance of mnesia
disc_only bag table, which is backed by dets. I used the following function
to generate records, where I can set the amount of repeating keys by
?REPCOUNT:

fill_modulo(_, Tab, 0, _, _, AccessMod) ->
	{ok, tab_size(Tab, AccessMod)};
fill_modulo(run, Tab, N, Max, T, AccessMod) ->
	T1 = progress(N, Max, T),
	R = #trec{ dbid = N rem ?REPCOUNT, data1 = <<"Hello World!">>, data2
= Max - N },
	case write_transaction(Tab, R, AccessMod) of
		ok ->
			fill_modulo(run, Tab, N-1, Max, T1, AccessMod);
		error ->
			{error, Max-N}
	end.  

This is the result of inserting 4M records with ?REPCOUNT set to 10000 (each
line shows the running time of the given 5% of the whole task):
......... 11.01s at 5%
......... 11.44s at 10%
......... 12.39s at 15%
......... 12.37s at 20%
......... 12.89s at 25%
......... 13.53s at 30%
......... 14.09s at 35%
......... 14.04s at 40%
......... 15.07s at 45%
......... 14.84s at 50%
......... 15.52s at 55%
......... 15.49s at 60%
......... 16.57s at 65%
......... 16.25s at 70%
......... 17.60s at 75%
......... 16.89s at 80%
......... 17.41s at 85%
......... 18.63s at 90%
......... 17.77s at 95%
.........
tab7 -> (ok) modulo insert into bag 4000000 of 4000000 in 306.931000s

And this is the result of the same test ran with ?REPCOUNT set to 100:
......... 30.34s at 5%
......... 70.68s at 10%
......... 117.20s at 15%
......... 163.29s at 20%
......... 203.08s at 25%
......... 246.20s at 30%
... and giving up waiting.

Since then I learned some other problems using mnesia/dets in this manner
(see my post 'mnesia silent failure and table corruption' and Ahmed Omar's
reply), and I stopped further investigation. Now I'm evaluating Berkely DB
for the task.

Best Regards,
Janos

-----Original Message-----
From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Dmitry Kolesnikov
Sent: Sunday, September 08, 2013 9:08 PM
To: erlang-questions Questions
Subject: Re: [erlang-questions] dets disk i/o

ping!
It looks that I should measure it by myself.

Best Regards, 
Dmitry

On Sep 4, 2013, at 12:09 AM, Dmitry Kolesnikov <dmkolesnikov@REDACTED>
wrote:

> Hello,
> 
> Does any one run a disk i/o analysis on dets tables?
> I am very curious about bag case, write scenario. 
> 
> Let's assume that I have a key which is associated with multiple entries.
> What is dets behaviour when I am writing a new entry to same key.
> Does it make append-only to data slot? or Does it read/write whole slot? 
> 
> Best Regards,
> Dmitry  

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions