[erlang-questions] Managing a huge binary file

Salvador Tamarit stamarit@REDACTED
Tue Apr 30 03:23:18 CEST 2013


hi all,

I have a very interesting problem for us :)

Suppose that we have a huge binary file (8GB or so) containing all the
integers from 0 to a given (very big) number. The numbers are unsorted
and some of them (70 numbers for instance) has been removed.

The goal is to find the the Nth removed number (in order). For
instance, if the file contains the numbers from 0 to 99, and then
number 101, the 1st removed would be 100.

I have been trying to solve this problem with different solutions, but
I always get stucked in reading the file. I'm reading it sequentally.
I open the file with:

file:open(FileName,[binary])

then I read each number with:

file:read(FDevice, 4)

this works fine, but too slow.

Anyway, supposing that I would be able to read it, then I don't know
how to manage the data (I think that it is impossible to load all this
data in memory).

How would you solve a problem like this?

Thanks,
Salva



More information about the erlang-questions mailing list