File sharing software
Mon May 22 19:57:26 CEST 2006
On May 3, 2006, at 6:18 PM, Claes Wikstrom wrote:
> Yariv Sadan wrote:
>> Maybe it's possible to develop for Mnesia an alternative disc
>> only storage engine designed for storing large blobs. It wouldn't
>> have to be an Oracle killer. It would just have to handle very
>> large data sets, have decent performance, and avoid the
>> rebuilding cost associated with broken dets tables. Such a
>> solution would make Mnesia more versatile and therefore more
>> appealing for applications for which it makes sense to sacrifice
>> soft real-time performance for large storage capacity. What do
>> you think?
> This is certainly possible, I wrote dets some 15 years ago, and
> it high time to rewrite completely. It's been through several
> enhancements within the OTP group, but the original ideas are still
> there. Linear (P.A Larsson algorithm) hashing for indexing and
> a buddy allocator for space on the file.
> Two major problems with dets as of today (and of yeasterday)
> 1. 64 bit indexing - really easy to fix
> 2. repair time - this is a bit harder to fix, but one solution
> could be along the following lines.
> a) keep the index and the freelist in a different file,
> b) have a file soley for the data. This file could be as easily
> repaired as a disklog file. One could just chunk through it, one
> term at a time, this way even large files will be fast to repair.
> c) when an object is put on the freelist, it needs also to be
> My daily 2c :-)
I have a few questions following up on Claes's comments. I'm still an
Erlang newbie (hopefully not for too long), so please bear with me if
they sound simple :)
- I'm a little confused about the 32 bit file offset issue. Integers
in Erlang can be greater than 2^32, so how come dets offsets are
limited to 32 bits? Is this a limitation of BEAM's native file IO code?
- Is dets limited to 32 bit offsets even if you run Erlang on 64 bit
- Is anybody planning on implementing Claes's solution to the repair
time issue described above?
Your answers are appreciated!
More information about the erlang-questions