[erlang-questions] Mnesia/dets question - does this make sense?

[Igor Ribeiro Sucupira]

Just another guess that's probably wrong: do you have write-back
caching enabled for your storage? If so, does disabling it make
writing times more uniform?

Igor.

On Fri, Sep 3, 2010 at 11:25 AM, Evans, Matthew <mevans@REDACTED> wrote:
> Possibly. In this case we are running on Linux, but the file system is EXT3 running on flash/SSD, so there shouldn't be any spinning disks that need to get moving.
>
> There are of course buffers down in the kernel space, but I imagine that they are global and not per file, so they shouldn't cause this behavior.
>
> It's not a huge issue, but could be if/when we need to grow the number of fragments.
>
> Matt
>
> ________________________________
> From: Nicholas Frechette [mailto:zeno490@REDACTED]
> Sent: Thursday, September 02, 2010 6:58 PM
> To: Evans, Matthew
> Cc: erlang-questions@REDACTED
> Subject: Re: [erlang-questions] Mnesia/dets question - does this make sense?
>
> Disc_only tables probably write to disk right away and probably don't buffer so writes can be atomic.
> I'm thinking your problem is more likely at the OS/HDD driver level. It is possible your OS/HDD is caching the data in memory for short amounts of time in order to speed up subsequent access to said data, once flushed you pay the full price of accessing that section of the disk again (worse if the disk had stopped spinning). For example, Windows 7 is actually quite good at this, using all available memory that isn't being used by applications to buffer IO (or something like that).
>
> Just a wild guess though, you'll have to profile to understand what is happening (perhaps at the kernel level to see how much time is spend actually writing the data by the kernel VS time spent in erlang code).
> 2cents
> Nicholas
> On Thu, Sep 2, 2010 at 5:33 PM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>> wrote:
> Hi,
>
> I have a fragmented disc_only mnesia table. During tests I have noticed that it appears that the access (insert) times to the fragment improve over time, and then slow down after a period of inactivity.
>
> Do the tables/fragments get opened and then closed based on a timer that would affect initial inert times?
>
> For example, the table has 10 fragments. I see insert times in the 4000us range for the first 10 to 15 inserts, and then insert times averaging 400us subsequently. If there is no activity on the table for a few minutes access times are again in the 4000us range for the first 10 to 15 inserts. As I said before it's almost like there is an overhead to get things started/warmed up (e.g. opening the fragments/files), once all the fragments are opened the speed is fine.
>
> Can anyone confirm this?
>
> Thanks
>
> Matt
>
>



-- 
"The secret of joy in work is contained in one word - excellence. To
know how to do something well is to enjoy it." - Pearl S. Buck.