[erlang-questions] very large key lookup

[Ulf Wiger (TN/EAB)]

> -----Original Message-----
> From: Jon Hancock [mailto:ke.han@REDACTED] 
> Sent: den 25 september 2006 13:29
> To: Ulf Wiger (TN/EAB)
> Cc: Erlang/OTP discussions
> Subject: Re: [erlang-questions] very large key lookup
> 
> The largest integer value in 64-bit erlang is what? 2^64? or 
> does erlang have some tricks for larger than this?  I tried 
> to look up this info but can't seem to find it.
> Any ideas on which tree or hash table to use for lookups?
> thanks, ke han

For hundreds of million keys, you might want to go with
a mnesia cluster. See the mnesia User Manual on 
fragmented tables.

http://www.erlang.org/doc/doc-5.5/lib/mnesia-4.3.1/doc/html/Mnesia_chap5
.html#5.3

(You can start with a non-fragmented table and grow into a fragmented
one.)

There is also some sample code in mnesia-xxxx/examples/bench/

That code actually implements a model of a HLR (Home Location Register)
which is the subscriber database of a mobile phone network. It 
does subscriber management/lookup based on primary key access.
Whether the key is of type binary or other doesn't matter.
The application scales very well when more processors are 
added, and the problem lends itself perfectly to a cluster 
database.

The 'bench' code uses several tricks, such as sticky locks.
If you can work your way through that small application,
I think you will have a good starting point for writing 
a very scalable application based on mnesia.

BR,
Ulf W