ets:select to ets

[Ulf Wiger]

When hacking the index functionality in Mnesia,
I was reminded of an idea I had a while ago.

When calling ets:select/2, it would sometimes be
very useful to be able to direct the result to
an ets table instead of the process heap. When
post-processing of the result is needed, one
either has to manage a large list on the process
heap, or one creates a temporary ets table.

In the latter case, copying the output of select()
directly to an ets table would cut down on the
copying, and also significantly reduce the GC issues.

Perhaps the following functions from mnesia_index.erl
can illustrate:

realkeys(Tab, Pos, IxKey) ->
     Index = get_index_table(Tab, Pos),
     db_get(Index, IxKey). % a list on the form [{IxKey, RealKey1} , ....

dirty_select(Tab, Spec, Pos) ->
     %% Assume that we are on the node where the replica is
     %% Returns the records without applying the match spec
     %% The actual filtering is handled by the caller
     IxKey = element(Pos, Spec),
     RealKeys = realkeys(Tab, Pos, IxKey),
     StorageType = val({Tab, storage_type}),
     lists:append([mnesia_lib:db_get(StorageType, Tab, Key) || {_,Key} <-  
RealKeys]).


What happens is that dirty_select produces a list of
lists (index values that each point to a list of objects.)
In order to return a flat list, mnesia builds a
(potentially large) list on the heap, and then calls
lists:append/1 on it (which essentially flattens it.)

In this case, I think it would be much more efficient
to direct the results of the ets operations to a
temporary ets table and then calling tab2list on it.

/Uffe