[erlang-questions] The 2 GB limit

[Igor Ribeiro Sucupira]

On Tue, Nov 24, 2009 at 12:47 PM, Valentin Micic <v@REDACTED> wrote:
> I must express my skepticism regarding dets performance when a file reaches
> such a level of "data density"

Hello.

Since I don't know very much about the inner workings of dets files,
maybe I am confused by the terminology: wouldn't data density be
inversely proportional to file size and directly proportional to data
size?

Thank you.
Igor.

> (assuming that 2 GB translates to at least a
> few million records). As much as I believe that reading from such a big file
> may be relatively fast, I think that updating the data would be
> disproportionately slower. In my experience, the only way to maintain a good
> performance with sizable data sets (say, 50-150 million records) in dets is
> to distribute data over a number of dets files (similar to what mnesia
> fragmentation does).
> So, if my skepticism is justifiable, this is not a question of how difficult
> it is to increase a dets file size limit, but would there be any point in
> doing it?
>
> V/
>
> -----Original Message-----
> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On
> Behalf Of Igor Ribeiro Sucupira
> Sent: 24 November 2009 04:24 PM
> To: erlang-questions
> Subject: [erlang-questions] The 2 GB limit
>
> Hi.
>
> Is there any plan (or work in progress) for removing the 2 GB size
> limit of dets files?
>
> And, while this is not accomplished, do you think it could be easier
> to try to raise that limit to 4 GB? Assuming the issue is related to
> 32-bit addressing, I'm guessing (just guessing) that dealing with 4 GB
> files should not be difficult. Am I wrong? Why?
>
> Thanks.
> Igor.
>
> --
> "The secret of joy in work is contained in one word - excellence. To
> know how to do something well is to enjoy it." - Pearl S. Buck.
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>



-- 
"The secret of joy in work is contained in one word - excellence. To
know how to do something well is to enjoy it." - Pearl S. Buck.