File sharing software

Yariv Sadan <>
Thu May 4 01:31:39 CEST 2006


Well, Claes, that's very reassuring! If you're the one who wrote  
dets, nobody would know better than you how to enhance it. Would you  
be interested in implementing this feature? (I suppose I could try to  
follow your guidelines and hack my own storage engine, but that would  
be Plan B :) )

Yariv

On May 3, 2006, at 6:18 PM, Claes Wikstrom wrote:

> Yariv Sadan wrote:
>
>> Maybe it's possible to develop for Mnesia an alternative disc  
>> only  storage engine designed for storing large blobs. It wouldn't  
>> have to  be an Oracle killer. It would just have to handle very  
>> large data  sets, have decent performance, and avoid the  
>> rebuilding cost  associated with broken dets tables. Such a  
>> solution would make Mnesia  more versatile and therefore more  
>> appealing for applications for  which it makes sense to sacrifice  
>> soft real-time performance for  large storage capacity. What do  
>> you think?
>>
>
>
> This is certainly possible, I wrote dets some 15 years ago, and
> it high time to rewrite completely. It's been through several
> enhancements within the OTP group, but the original ideas are still
> there. Linear (P.A Larsson algorithm) hashing for indexing and
> a buddy allocator for space on the file.
>
> Two major problems with dets as of today (and of yeasterday)
>
> 1. 64 bit indexing - really easy to fix
> 2. repair time - this is a bit harder to fix, but one solution
>    could be along the following lines.
>
>   a) keep the index and the freelist in a different file,
>   b) have a file soley for the data. This file could be as easily
>      repaired as a disklog file. One could just chunk through it, one
>      term at a time, this way even large files will be fast to repair.
>   c) when an object is put on the freelist, it needs also to be
>      overwriten
>
>
> My daily 2c :-)
>
>
> /klacke
>
>
> -- 
> Claes Wikstrom                        -- Caps lock is nowhere and
> http://www.hyber.org                  -- everything is under control
> cellphone: +46 70 2097763




More information about the erlang-questions mailing list