[erlang-questions] Mnesia normalized field lookup

[Szepes Tamás]

Thanks, but I rather thought about a library that automates these lookups
behind the scene. Maybe through a customized activity access callback module
or somehow with the traverse parameter of mnesia:table/2.

 

Tamas 

 

From: Chandru [mailto:chandrashekhar.mullaparthi@REDACTED] 
Sent: Monday, January 28, 2013 12:04 AM
To: Szepes Tamás
Cc: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Mnesia normalized field lookup

 

You could consider computing the hash of your value, and store the mapping
from actual value to hash value in another table. When writing your record,
store the hash value instead of the actual value. When you need to retrieve
the original, use the mapping table to map the hashed value to the actual
value. Does that make sense?

Chandru

On 27 January 2013 18:00, Szepes Tamás <tamas@REDACTED> wrote:

All,

I have couple of Mnesia tables where the records has some long text fields
with relatively little variation in content (like institute_name which can
be 128 characters long, but in a typical installation there is no more than
20 different values). About 1/3 of my fields are similar and I cannot
determine all possible values in advance to make atoms for them (data from a
new institute can come any time after installation, and the database needs
to be self contained for archiving purposes).
As I need to handle ~10^8 records I’d like to normalize these fields to
spare storage space. However it blows my code if I have to do 3-6 joins in
every lookup operation, not to mention the inserts.
Is there any elegant way of handling this situation?

Thanks in advance,
Tamas

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130128/8e551ac2/attachment.htm>