<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=iso-8859-1"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Courier New";
color:#202020;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New";color:#202020'>Thanks, but I rather thought about a library that automates these lookups behind the scene. Maybe through a customized activity access callback module or somehow with the traverse parameter of mnesia:table/2.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New";color:#202020'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New";color:#202020'>Tamas <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New";color:#202020'><o:p> </o:p></span></p><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Chandru [mailto:chandrashekhar.mullaparthi@gmail.com] <br><b>Sent:</b> Monday, January 28, 2013 12:04 AM<br><b>To:</b> Szepes Tamás<br><b>Cc:</b> erlang-questions@erlang.org<br><b>Subject:</b> Re: [erlang-questions] Mnesia normalized field lookup<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'>You could consider computing the hash of your value, and store the mapping from actual value to hash value in another table. When writing your record, store the hash value instead of the actual value. When you need to retrieve the original, use the mapping table to map the hashed value to the actual value. Does that make sense?<br><br>Chandru<o:p></o:p></p><div><p class=MsoNormal>On 27 January 2013 18:00, Szepes Tamás <<a href="mailto:tamas@veriport.eu" target="_blank">tamas@veriport.eu</a>> wrote:<o:p></o:p></p><p class=MsoNormal>All,<br><br>I have couple of Mnesia tables where the records has some long text fields<br>with relatively little variation in content (like institute_name which can<br>be 128 characters long, but in a typical installation there is no more than<br>20 different values). About 1/3 of my fields are similar and I cannot<br>determine all possible values in advance to make atoms for them (data from a<br>new institute can come any time after installation, and the database needs<br>to be self contained for archiving purposes).<br>As I need to handle ~10^8 records I’d like to normalize these fields to<br>spare storage space. However it blows my code if I have to do 3-6 joins in<br>every lookup operation, not to mention the inserts.<br>Is there any elegant way of handling this situation?<br><br>Thanks in advance,<br>Tamas<br><br>_______________________________________________<br>erlang-questions mailing list<br><a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br><a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div></body></html>