[erlang-questions] ets with maps and/or select

[zxq9]

On 2016年5月19日 木曜日 11:16:43 Matthew Evans wrote:
> Hi,
> I want to create an ets table to store a large number of records (over 100,000). I then want to use ets:select/ets:match to search this table for records containing a specific value. The issue is that I do not know the position of this value in the record. For example take this record:
> R1#my_record{key = 123, value1 = 77, value2 = 88}
> R2#my_record{key = 321, value1 = 88, value2 = 77}
> R3#my_record{key = 999, value1 = 12, value2 = 73}
> I want to search the ets table to find any record containing value 77 (I could of course save these values in a list in each record, this is just an example).
> I know I can do an ets:foldl for this purpose, but I was wondering if there is a more efficient way?
> Could I for example save the values in a map, put the map in a record in the ets table and create a match_specification that I can use with an ets:select or ets:match?
> The map may look like:
> M = #{77 => ok, 88 => ok}.R4#my_record{key = 123, map = M}.

This is the kind of thing you would build an index for. You could create a K/V structure (map, dict, another ETS table, whatever) of the form {Value, [AssociatedRecordKeys]}. The query steps then become:
1- Check the index
2- Directly pull the list of records stored in ETS associated with that value.

This is, of course, doing half the work of maintaining an index yourself, which is a databasey thing to be doing.

If its just this one task in your whole project then the overhead of maintaining the index and mental overhead of writing it is probably lower than including a database in your project. But if you have even one more situation that seems similar to this then the better use of your time would probably be to include a DB product that provides this kind of indexing out of the box and using that rather than building your ownDB-ish indexing thingy out of lower-level data structures/operations.

I think.
Maybe R19 has some magical operation I don't know about. ...?

-Craig