[erlang-questions] Large array of data structures

[David Mitchell]

I guess the obvious response would be "use a database" for that
quantity of data, but I can't see that you're going to get 2ms search
times once you write it to disc.  Pretty sure that limits you to
RAM-only solutions.

Anyone know how well mnesia scales up in that scenario?

Regards

Dave M.

On 19/09/2007, Dmitrii 'Mamut' Dimandt <dmitriid@REDACTED> wrote:
> I've received a question over at "Erlang in Russian" forum,
> http://erlang.dmitriid.com/forum/topic/11 that reads:
> "
> I need to process a huge array of structures - about 60 million of them
> in total. Each structure looks like this:  {PID, type (atom), direct
> (atom), index (Integer), nextind (Integer), prevind (Integer), position
> (Integer)}. I think such a structure would take up about 50 bytes and
> the entire array would then need 60 * 50 = 3 GB. It means that the array
> must be partly written to disk. In addition I need a fast access to PID,
> index, nextind, prevind fields. Searching for a structure by these
> fields should take about 2 microseconds, saving such a structure should
> takle as much. Reading and writing is performed on the same structures
> (about 30 000 of them) while the others simply wait. So the question is:
> how can i implement such an array?
> "
>
>
> My guess is that (d)ets/mnesia could be involved, but that's all I know
> :) Can anyone help?
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>