[erlang-questions] Mnesia normalized field lookup
Mon Jan 28 22:36:50 CET 2013
Thanks, but I rather thought about a library that automates these lookups
behind the scene. Maybe through a customized activity access callback module
or somehow with the traverse parameter of mnesia:table/2.
From: Chandru [mailto:]
Sent: Monday, January 28, 2013 12:04 AM
To: Szepes Tamás
Subject: Re: [erlang-questions] Mnesia normalized field lookup
You could consider computing the hash of your value, and store the mapping
from actual value to hash value in another table. When writing your record,
store the hash value instead of the actual value. When you need to retrieve
the original, use the mapping table to map the hashed value to the actual
value. Does that make sense?
On 27 January 2013 18:00, Szepes Tamás <> wrote:
I have couple of Mnesia tables where the records has some long text fields
with relatively little variation in content (like institute_name which can
be 128 characters long, but in a typical installation there is no more than
20 different values). About 1/3 of my fields are similar and I cannot
determine all possible values in advance to make atoms for them (data from a
new institute can come any time after installation, and the database needs
to be self contained for archiving purposes).
As I need to handle ~10^8 records Id like to normalize these fields to
spare storage space. However it blows my code if I have to do 3-6 joins in
every lookup operation, not to mention the inserts.
Is there any elegant way of handling this situation?
Thanks in advance,
erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions