Emulator stopping during mnesia writes

[Klacke]

On Wed, May 10, 2000 at 05:33:08PM +0100, Sean Hinde wrote:
> Thanks Scott,
> 
> > If indeed the disk/disks storing the file system with the mnesia logs
> > is too busy, the solutions are few: software-based striping across
> > multiple disk drives, hardware-based striping across multiple disk
> > drives, solid state disk drives, or algorithmic changes to reduce I/O
> > workload.  The last may be more difficult to do than the former
> > three.  :-)
> 
> I am hoping that there may be some fix in the future where disk i/o is
> farmed off to a separate thread/threads within the emulator. Then if i/o
> blocks it will not stop servicing other requests which are for RAM based
> data. There is a rumour something like this MAY be in R7...
> 
> One possible solution I am looking at is to have a set of nodes with
> disc_only_copies which just hold the persistent data, and a separate set of
> nodes with ram_copies to service the database reads. Other suggestions
> welcome.
> 

We at bluetail have had some serious troble with disk io too. In
particular loads of disk operations that modify a directory entry 
such as open/creat, unlink, rename

What makes the situation a lot worse is that while disk io is
pending, the emulator can't run no reductions at all and the overall 
performance drops to an almost grinding halt.

It also turns out that different file systems are better or worse.
In particular FreeBSD with Soft updates performed well whereas
UFS/Solaris and FFS/BSD performed bad (They are almost identical)

We have a short term kludge solution (in production use today) which
is an external unix process that offloads the emulator
with calls to unlink,stat,link and rename.

In the longer run we have an async driver in the pipe 
which schedules job to kernel threads. (if the os have them) 

This async driver might just make it into r7 this fall. 

However if the applictaion is soley io bound, the overall 
performance is of corce soley dependant on the underlying 
disk system, this is however most often not the typical case.
Typically an application has lots of stuff to do, it's just that
sometimes disk io becomes very intesive (mnesia dump for example)
and in those cases it would be nice if some other thread/process than
the emulator is suspended waiting for io completion.


/klacke


-- 
Claes Wikstrom                          <klacke@REDACTED>
Bluetail AB                             http://www.bluetail.com