[erlang-questions] Mnesia/dets question - does this make sense?

[Evans, Matthew]

Possibly. In this case we are running on Linux, but the file system is EXT3 running on flash/SSD, so there shouldn't be any spinning disks that need to get moving.

There are of course buffers down in the kernel space, but I imagine that they are global and not per file, so they shouldn't cause this behavior.

It's not a huge issue, but could be if/when we need to grow the number of fragments.

Matt

________________________________
From: Nicholas Frechette [mailto:zeno490@REDACTED]
Sent: Thursday, September 02, 2010 6:58 PM
To: Evans, Matthew
Cc: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Mnesia/dets question - does this make sense?

Disc_only tables probably write to disk right away and probably don't buffer so writes can be atomic.
I'm thinking your problem is more likely at the OS/HDD driver level. It is possible your OS/HDD is caching the data in memory for short amounts of time in order to speed up subsequent access to said data, once flushed you pay the full price of accessing that section of the disk again (worse if the disk had stopped spinning). For example, Windows 7 is actually quite good at this, using all available memory that isn't being used by applications to buffer IO (or something like that).

Just a wild guess though, you'll have to profile to understand what is happening (perhaps at the kernel level to see how much time is spend actually writing the data by the kernel VS time spent in erlang code).
2cents
Nicholas
On Thu, Sep 2, 2010 at 5:33 PM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>> wrote:
Hi,

I have a fragmented disc_only mnesia table. During tests I have noticed that it appears that the access (insert) times to the fragment improve over time, and then slow down after a period of inactivity.

Do the tables/fragments get opened and then closed based on a timer that would affect initial inert times?

For example, the table has 10 fragments. I see insert times in the 4000us range for the first 10 to 15 inserts, and then insert times averaging 400us subsequently. If there is no activity on the table for a few minutes access times are again in the 4000us range for the first 10 to 15 inserts. As I said before it's almost like there is an overhead to get things started/warmed up (e.g. opening the fragments/files), once all the fragments are opened the speed is fine.

Can anyone confirm this?

Thanks

Matt