[erlang-questions] Large array of data structures

Dmitrii 'Mamut' Dimandt <>
Tue Sep 18 17:44:04 CEST 2007


I've received a question over at "Erlang in Russian" forum,
http://erlang.dmitriid.com/forum/topic/11 that reads:
"
I need to process a huge array of structures - about 60 million of them
in total. Each structure looks like this:  {PID, type (atom), direct
(atom), index (Integer), nextind (Integer), prevind (Integer), position
(Integer)}. I think such a structure would take up about 50 bytes and
the entire array would then need 60 * 50 = 3 GB. It means that the array
must be partly written to disk. In addition I need a fast access to PID,
index, nextind, prevind fields. Searching for a structure by these
fields should take about 2 microseconds, saving such a structure should
takle as much. Reading and writing is performed on the same structures
(about 30 000 of them) while the others simply wait. So the question is:
how can i implement such an array?
"


My guess is that (d)ets/mnesia could be involved, but that's all I know
:) Can anyone help?



More information about the erlang-questions mailing list