Handling huge amounts of data

[Vlad Dumitrescu]

Hi,

I am writing some programs that need to shuffle a lot of data (arrays/lists
of some 5.000.000 items). I didn't want to use C/C++, so I settled to C#.
None of the above, however, are especially good prototyping.

I would have loved to be able to give Erlang a chance, but some simple tests
say most of the time goes to garbage collection (and I need some speed when
running too, not only when developing). Also, the memory usage seems to
always go up (except when a process ends and that memory is freed)[*].
Besides that, it's occasionally causing "Abnormal termination"s.

Is there some "magic" that will make Erlang work with this kind of problems,
or is it just not the right niche?

If the answer above is "no", has anyone any suggestion about what wo use
instead?

thanks in advance. Best regards,
Vlad

[*] Strange enough, there was a memory "up" when running "f(A):" in the
shell, A being a huge list... Why would tha happen?