<div dir="ltr">I have a table of type set that worked fine as a non-fragmented table. We started seeing performance issues with the table and with the anticipation of more clients in the system, we were worried the size would exceed the 2 GB max. <div>
<br></div><div>The table contains chat messages and we purge records older than 7 days. There seem to be a number of keys that don't show up in a read activity but, do show up when doing a table dump, a select, or using qlc to find records. As such, when this purge policy runs records are found that when I try to delete don't actually get deleted. It almost seems like read/delete are working off of one set while qlc is working off of another set.</div>
<div><br></div><div>Code/outputs below. As you can see for a given key read returns nothing, qlc returns an object, delete says it worked ok. Running the code again gives the same results. Am I using the fragmentation API incorrectly somehow?</div>
<div><br></div><div>Thanks in advance</div><div><br></div><div><b>Code:</b><br><div>-module(sel).</div><div>-include_lib("stdlib/include/qlc.hrl").</div><div>-record(archive_message313, {owner_with_day_utc, owner, with, day_gregorian_seconds, messages = []}).</div>
<div>-export([run/0]).</div><div><br></div><div>run () -></div><div> Owner = {"9afa8671-7e88-436c-9ad4-4cb13ad45e1e", "<a href="http://dj.barker.xxx.com">dj.barker.xxx.com</a>"},</div><div> With = {"d735280e-d263-4e2b-beff-ed8ccaca5535", "<a href="http://dj.barker.xxx.com">dj.barker.xxx.com</a>"}, </div>
<div> DayUtc = {2013, 6, 28}, </div><div> K = {Owner, With, DayUtc},</div><div><br></div><div> DelFun = fun () -> mnesia:delete({archive_message313, K}) end,</div><div> ReadFun = fun () -> mnesia:read({archive_message313, K}) end,</div>
<div><br></div><div> QlcFun = fun () -></div><div> Q = qlc:q([M || M <- mnesia:table(archive_message313), M#archive_message313.owner_with_day_utc == K]), </div><div> qlc:eval(Q)</div><div> end,</div><div> </div>
<div> Read = mnesia:activity(transaction, ReadFun, [], mnesia_frag),</div><div> Qlc = mnesia:activity(transaction, QlcFun, [], mnesia_frag),</div><div> Del = mnesia:activity(transaction, DelFun, [], mnesia_frag),</div>
<div><br></div><div> {read_result, Read, qlc_result, Qlc, del_result, Del}.</div><div><br></div><div><b>Output:</b></div><div><div>{read_result,[],qlc_result,</div><div> [{archive_message313,{{"9afa8671-7e88-436c-9ad4-4cb13ad45e1e",</div>
<div> "<a href="http://dj.barker.xxx.com">dj.barker.xxx.com</a>"},</div><div> {"d735280e-d263-4e2b-beff-ed8ccaca5535",</div><div> "<a href="http://dj.barker.xxx.com">dj.barker.xxx.com</a>"},</div>
<div> {2013,6,28}},</div><div> {"9afa8671-7e88-436c-9ad4-4cb13ad45e1e",</div><div> "<a href="http://dj.barker.xxx.com">dj.barker.xxx.com</a>"},</div>
<div> {"d735280e-d263-4e2b-beff-ed8ccaca5535",</div><div> "<a href="http://dj.barker.xxx.com">dj.barker.xxx.com</a>"},</div><div> 63539596800,</div>
<div> [{{{2013,6,28},{13,36,34}},</div><div> {xmlelement,"forwarded",</div><div> [{"xmlns","urn:xmpp:forward:0"}],</div>
<div> [{xmlelement,"delay",</div><div> [{"xmlns","urn:xmpp:delay"},</div><div> {"stamp","2013-06-28T13:36:34.000000Z"}],</div>
<div> []},</div><div> {xmlelement,"message",</div><div> [{"to",</div>
<div> "<a href="mailto:9afa8671-7e88-436c-9ad4-4cb13ad45e1e@dj.barker.xxx.com">9afa8671-7e88-436c-9ad4-4cb13ad45e1e@dj.barker.xxx.com</a>"},</div><div>
{"from",</div><div> "<a href="mailto:d735280e-d263-4e2b-beff-ed8ccaca5535@dj.barker.xxx.com">d735280e-d263-4e2b-beff-ed8ccaca5535@dj.barker.xxx.com</a>"},</div>
<div> {"type","chat"}],</div><div> [{xmlelement,"body",[],[...]}]}]},</div>
<div> 1372426594.889164},</div><div> {{{2013,6,28},{13,36,35}},</div><div> {xmlelement,"forwarded",</div><div>
[{"xmlns","urn:xmpp:forward:0"}],</div><div> [{xmlelement,"delay",</div><div> [{"xmlns","urn:xmpp:delay"},</div>
<div> {"stamp","2013-06-28T13:36:35.000000Z"}],</div><div> []},</div><div> {xmlelement,"message",</div>
<div> [{"to",</div><div> "<a href="mailto:9afa8671-7e88-436c-9ad4-4cb13ad45e1e@dj.barker.xxx.com">9afa8671-7e88-436c-9ad4-4cb13ad45e1e@dj.barker.xxx.com</a>"},</div>
<div> {"from",</div><div> "<a href="mailto:d735280e-d263-4e2b-beff-ed8ccaca5535@dj.barker.xxx.com">d735280e-d263-4e2b-beff-ed8ccaca5535@dj.barker.xxx.com</a>"},</div>
<div> {"type",[...]}],</div><div> [{xmlelement,"body",[],...}]}]},</div><div> 1372426595.124266},</div>
<div> {{{2013,6,28},{13,36,35}},</div><div> {xmlelement,"forwarded",</div><div> [{"xmlns","urn:xmpp:forward:0"}],</div>
<div> [{xmlelement,"delay",</div><div> [{"xmlns","urn:xmpp:delay"},</div><div> {"stamp","2013-06-28T13:36:35.000000Z"}],</div>
<div> []},</div><div> {xmlelement,"message",</div><div> [{"to",</div>
<div> "<a href="mailto:9afa8671-7e88-436c-9ad4-4cb13ad45e1e@dj.barker.xxx.com">9afa8671-7e88-436c-9ad4-4cb13ad45e1e@dj.barker.xxx.com</a>"},</div><div>
{"from",[...]},</div><div> {[...],...}],</div><div> [{xmlelement,[...],...}]}]},</div>
<div> 1372426595.337535}]}],</div><div> del_result,ok}</div></div>-- <br>Noah
</div></div>