[erlang-questions] ets:select badarg question (R15B)

[Matthew Evans]

Many thanks,
I'll give that a go. I am however a little worried about having a safe_fixtable "active" on a table for 900 seconds so might need to be a little smarter.
Cheers
Matt

> Date: Mon, 15 Oct 2012 14:50:34 +0200
> From: sverker.eriksson@REDACTED
> To: mattevans123@REDACTED
> CC: erlang-questions@REDACTED
> Subject: Re: [erlang-questions] ets:select badarg question (R15B)
> 
> You need to use ets:safe_fixtable/2 to get a "sensible" result from 
> select/match with continuation on unordered tables. Otherwise you may 
> miss records, get double hits and (as in your case) get badarg if the 
> table has shrunken due to deleted record causing the continuation to 
> point out of table bounds. One could argue that '$end_of_table' would be 
> a more suitable result than badarg for this case.
> 
> I see that the documentation is not clear about this. It describes usage 
> of safe_fixtable together with first/1 and next/2 but it does not state 
> that it also is needed for select/match with continuation.
> Single call select/match does not need safe_fixtable.
> 
> /Sverker, Erlang/OTP
> 
> Matthew Evans wrote:
> > Hi
> > These are the options
> >
> >
> > ets:new(StationIdTableName,[{keypos,2},named_table,public]),
> > The only odd things I can think of is that during the select records are getting deleted and added. Also, the select operation is run slowly (over 900 seconds)
> > Again, this isn't every time I get the crash.
> > Matt
> >
> >   
> >> Date: Mon, 15 Oct 2012 11:49:14 +0200
> >> From: sverker.eriksson@REDACTED
> >> To: mattevans123@REDACTED
> >> CC: erlang-questions@REDACTED
> >> Subject: Re: [erlang-questions] ets:select badarg question (R15B)
> >>
> >> What options are you using when the table is created?
> >>
> >> /Sverker, Erlang/OTP
> >>
> >> Matthew Evans wrote:
> >>     
> >>> Hi,
> >>> I'm running a select with a continuation on a public ets table with approximately 20,000 records.
> >>> The original call to the select is: ets:select(TableName,[{'$1',[],['$_']}],RecordsToGet), then an ets:select/1 on the continuation.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> The process that is spawned to walk the table is sitting in a recursive loop doing a select (grabbing about 150 records each time), processing the matching records (which includes possibly deleting some of these records), then sleeping for a few seconds. It takes about 900 seconds to walk through the table. As well as records getting deleted, new records are added to the table during this time.
> >>> Every now and then I get an exception in the process doing the select/1:
> >>> Error in process <0.16218.1> on node 'xxxxx@REDACTED' with exit value: {badarg,[{ets,select,[{'table_E0:39:D7:00:11:80',301,6,<<0 bytes>>,[{p_MacEntry.......  (truncated here)
> >>> This is the code (all running in the same process that started the select) obviously the last ets:select is causing the crash:
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> start_aging1(AgeInterval,MaxAge,SelectData) ->
> >>>      start_aging1(AgeInterval,MaxAge,SelectData,0).
> >>> start_aging1(_AgeInterval,_MaxAge,'$end_of_table',RecsAged) ->
> >>>      gen_server:cast(?MODULE,{completed_aging_operation,RecsAged});
> >>> start_aging1(_AgeInterval,MaxAge,{Matches,'$end_of_table'},RecsAged) ->
> >>>      RecordsAgedCount = age_records(Matches,[],MaxAge*1000000,os:timestamp()),
> >>>      gen_server:cast(?MODULE,{completed_aging_operation,RecsAged+RecordsAgedCount});
> >>> start_aging1(AgeInterval,MaxAge,{Matches,Continuation},RecsAged) ->
> >>>      RecordsAgedCount = age_records(Matches,[],MaxAge*1000000,os:timestamp()),
> >>>      timer:sleep(AgeInterval),
> >>>      start_aging1(AgeInterval,MaxAge,ets:select(Continuation),RecsAged+RecordsAgedCount).
> >>>
> >>> I'm not sure what could be the cause of the select getting a bad arg.
> >>> Any ideas?
> >>> Thanks
> >>> Matt                                             		 	   		  
> >>>   
> >>> ------------------------------------------------------------------------
> >>>
> >>> _______________________________________________
> >>> erlang-questions mailing list
> >>> erlang-questions@REDACTED
> >>> http://erlang.org/mailman/listinfo/erlang-questions
> >>>   
> >>>       
> >  		 	   		  
> >   
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121015/f65422d5/attachment.htm>