[erlang-questions] ets:select badarg question (R15B)

[Sverker Eriksson]

You need to use ets:safe_fixtable/2 to get a "sensible" result from 
select/match with continuation on unordered tables. Otherwise you may 
miss records, get double hits and (as in your case) get badarg if the 
table has shrunken due to deleted record causing the continuation to 
point out of table bounds. One could argue that '$end_of_table' would be 
a more suitable result than badarg for this case.

I see that the documentation is not clear about this. It describes usage 
of safe_fixtable together with first/1 and next/2 but it does not state 
that it also is needed for select/match with continuation.
Single call select/match does not need safe_fixtable.

/Sverker, Erlang/OTP

Matthew Evans wrote:
> Hi
> These are the options
>
>
> ets:new(StationIdTableName,[{keypos,2},named_table,public]),
> The only odd things I can think of is that during the select records are getting deleted and added. Also, the select operation is run slowly (over 900 seconds)
> Again, this isn't every time I get the crash.
> Matt
>
>   
>> Date: Mon, 15 Oct 2012 11:49:14 +0200
>> From: sverker.eriksson@REDACTED
>> To: mattevans123@REDACTED
>> CC: erlang-questions@REDACTED
>> Subject: Re: [erlang-questions] ets:select badarg question (R15B)
>>
>> What options are you using when the table is created?
>>
>> /Sverker, Erlang/OTP
>>
>> Matthew Evans wrote:
>>     
>>> Hi,
>>> I'm running a select with a continuation on a public ets table with approximately 20,000 records.
>>> The original call to the select is: ets:select(TableName,[{'$1',[],['$_']}],RecordsToGet), then an ets:select/1 on the continuation.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> The process that is spawned to walk the table is sitting in a recursive loop doing a select (grabbing about 150 records each time), processing the matching records (which includes possibly deleting some of these records), then sleeping for a few seconds. It takes about 900 seconds to walk through the table. As well as records getting deleted, new records are added to the table during this time.
>>> Every now and then I get an exception in the process doing the select/1:
>>> Error in process <0.16218.1> on node 'xxxxx@REDACTED' with exit value: {badarg,[{ets,select,[{'table_E0:39:D7:00:11:80',301,6,<<0 bytes>>,[{p_MacEntry.......  (truncated here)
>>> This is the code (all running in the same process that started the select) obviously the last ets:select is causing the crash:
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> start_aging1(AgeInterval,MaxAge,SelectData) ->
>>>      start_aging1(AgeInterval,MaxAge,SelectData,0).
>>> start_aging1(_AgeInterval,_MaxAge,'$end_of_table',RecsAged) ->
>>>      gen_server:cast(?MODULE,{completed_aging_operation,RecsAged});
>>> start_aging1(_AgeInterval,MaxAge,{Matches,'$end_of_table'},RecsAged) ->
>>>      RecordsAgedCount = age_records(Matches,[],MaxAge*1000000,os:timestamp()),
>>>      gen_server:cast(?MODULE,{completed_aging_operation,RecsAged+RecordsAgedCount});
>>> start_aging1(AgeInterval,MaxAge,{Matches,Continuation},RecsAged) ->
>>>      RecordsAgedCount = age_records(Matches,[],MaxAge*1000000,os:timestamp()),
>>>      timer:sleep(AgeInterval),
>>>      start_aging1(AgeInterval,MaxAge,ets:select(Continuation),RecsAged+RecordsAgedCount).
>>>
>>> I'm not sure what could be the cause of the select getting a bad arg.
>>> Any ideas?
>>> Thanks
>>> Matt                                             		 	   		  
>>>   
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>   
>>>       
>  		 	   		  
>