Handling huge amounts of data
Thu Jun 5 14:39:10 CEST 2003
First things first, I have a programm running, running in Delphi. Doing the
tests for one item takes some 30s, and since I have to choose at least 50000
of them, it's going to be running some 3 weeks. I guess any interpreted
language will be slower than direct memory access. The good news is it's
going to be run once a year, or so.
> Hmm, interesting that it is always the same size. Can you say more
> about the test? Is there any similarity in the data content? Any way
> to compress the data that is related to the search mechanism? Does
> the problem map to other similar problems? How many search requests
> do you get and how often? How long do you have to respond (latency
> and thruput)?
I am not sure if everybody is interested in more details, and also I am not
allowed to say too much, but the answer to most of the questions above is
'no, as far as I can tell'. The items are the numbers 1 to 27 and I have to
filter out certain interdependencies, and the comparisons need to be done
one by one.
Thanks for all the good ideas. I'll look for another opportunity to use
More information about the erlang-questions