File sharing software

[Claes Wikstrom]

Yariv Sadan wrote:

> 
> 
> Maybe it's possible to develop for Mnesia an alternative disc only  
> storage engine designed for storing large blobs. It wouldn't have to  be 
> an Oracle killer. It would just have to handle very large data  sets, 
> have decent performance, and avoid the rebuilding cost  associated with 
> broken dets tables. Such a solution would make Mnesia  more versatile 
> and therefore more appealing for applications for  which it makes sense 
> to sacrifice soft real-time performance for  large storage capacity. 
> What do you think?
>


This is certainly possible, I wrote dets some 15 years ago, and
it high time to rewrite completely. It's been through several
enhancements within the OTP group, but the original ideas are still
there. Linear (P.A Larsson algorithm) hashing for indexing and
a buddy allocator for space on the file.

Two major problems with dets as of today (and of yeasterday)

1. 64 bit indexing - really easy to fix
2. repair time - this is a bit harder to fix, but one solution
    could be along the following lines.

   a) keep the index and the freelist in a different file,
   b) have a file soley for the data. This file could be as easily
      repaired as a disklog file. One could just chunk through it, one
      term at a time, this way even large files will be fast to repair.
   c) when an object is put on the freelist, it needs also to be
      overwriten


My daily 2c :-)


/klacke


-- 
Claes Wikstrom                        -- Caps lock is nowhere and
http://www.hyber.org                  -- everything is under control
cellphone: +46 70 2097763