File sharing software

[Yariv Sadan]

Well, Claes, that's very reassuring! If you're the one who wrote  
dets, nobody would know better than you how to enhance it. Would you  
be interested in implementing this feature? (I suppose I could try to  
follow your guidelines and hack my own storage engine, but that would  
be Plan B :) )

Yariv

On May 3, 2006, at 6:18 PM, Claes Wikstrom wrote:

> Yariv Sadan wrote:
>
>> Maybe it's possible to develop for Mnesia an alternative disc  
>> only  storage engine designed for storing large blobs. It wouldn't  
>> have to  be an Oracle killer. It would just have to handle very  
>> large data  sets, have decent performance, and avoid the  
>> rebuilding cost  associated with broken dets tables. Such a  
>> solution would make Mnesia  more versatile and therefore more  
>> appealing for applications for  which it makes sense to sacrifice  
>> soft real-time performance for  large storage capacity. What do  
>> you think?
>>
>
>
> This is certainly possible, I wrote dets some 15 years ago, and
> it high time to rewrite completely. It's been through several
> enhancements within the OTP group, but the original ideas are still
> there. Linear (P.A Larsson algorithm) hashing for indexing and
> a buddy allocator for space on the file.
>
> Two major problems with dets as of today (and of yeasterday)
>
> 1. 64 bit indexing - really easy to fix
> 2. repair time - this is a bit harder to fix, but one solution
>    could be along the following lines.
>
>   a) keep the index and the freelist in a different file,
>   b) have a file soley for the data. This file could be as easily
>      repaired as a disklog file. One could just chunk through it, one
>      term at a time, this way even large files will be fast to repair.
>   c) when an object is put on the freelist, it needs also to be
>      overwriten
>
>
> My daily 2c :-)
>
>
> /klacke
>
>
> -- 
> Claes Wikstrom                        -- Caps lock is nowhere and
> http://www.hyber.org                  -- everything is under control
> cellphone: +46 70 2097763