[erlang-questions] Needed: Great big ordered int set.
Mon Jul 8 19:22:44 CEST 2013
The data structure is a sorted list of integers. That 500M dataset number, is that over a single list of integers, or is that the sum of all lists of integers?
What are the reliability requirements? Do you need redundancy and/or backups? It is a very different problem if a single server solution is enough, or if it requires a network of computers.
On Jul 8, 2013, at 7:11 PM, Alex Arnon wrote:
> Hi All,
> I need to implement a very large set of data, with the following requirements:
> - It will be populated EXCLUSIVELY by 64-bit integers.
> - The only operations will be:
> - add element,
> - get number of elements, and
> - fold/foreach over the SORTED dataset.
> - The invocation order will be strictly:
> - create data structure,
> - add elements sequentially,
> - run one or more iteration operations,
> - discard data structure.
> - The size of the dataset MUST scale to 500M elements, preferably billions should be possible too.
> - The data does not have to reside in memory - however, 32 to 64 GB of RAM may be allocated. (of course, these will be used by the OS buffer cache in case a file-based solution is chosen).
> In summary: Performance is not a must, but volume and the ability to iterate over the ordered values is.
> Thanks in advance!!!
> erlang-questions mailing list
More information about the erlang-questions