[erlang-questions] is storing complex objects in ets a goodidea?

[Ulf Wiger (TN/EAB)]

The organization I had in mind was to make sure the whole hierarchy
is visible in the key:
 
{{Line,Key,ColumnKey}, Value}
 
Then, if you know the line and key values, the following command
will efficiently retrieve all column key instances:
 
ets:match_object(Tab, {{Line, Key, '_'}, '_'}).
 
The cost will be roughly proportional to the number of objects returned,
by the search - not (so much) to the number of objects in the table,
even though the table size does affect the timings slightly.
 
If you don't know the Line, then searching will be linear, but this 
is the case with your dict of dicts as well.
 
The doc states that select is a more general form of match() and
match_object(),
so you need to follow the references back to match/2 and match_object/2 
where the following is written:
 
"If the key is specified in the pattern, the match is very efficient. If
the key is not specified, i.e. if it is a variable or an underscore, the
entire table must be searched. The search time can be substantial if the
table is very large."
 
It doesn't state that even a partially bound key gives an efficient
match.
 
BR,
Ulf W

________________________________

	From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Ludovic
Coquelle
	Sent: den 16 april 2007 09:02
	To: Ulf Wiger
	Cc: Erlang
	Subject: Re: [erlang-questions] is storing complex objects in
ets a goodidea?
	
	
	Thanks.
	Your answer make me understand than ets is more like a DB than
like a dict.
	
	So if my current matrix representation is something like:
	{line key, dict-of-column-values}
	a new implementation with ets should split the dict-of-column
structure: 
	{line key, column key 1, value}
	{line key, column key 2, value}
	...
	
	Btw, it was not clear to me from the doc than select and lookup
imply different efficiency results.
	
	Thank you very much.
	
	
	On 4/16/07, Ulf Wiger <ulf@REDACTED> wrote: 

		2007/4/16, Ludovic Coquelle <lcoquelle@REDACTED>: 
		
		

			Hi,
			
			In my application, I have to store a huge (but
sparse) 2D matrix. I implemented that successfully using dict-of-dict
kind of structure.
			
			I was then thinking to use a ets-of-dict
structure: each line is a dict (containing all colum values) and I store
each line (a dict) as an entry of a ets structure. 
			But I read in the doc introduction of ets that
"In the current implementation, every object insert and look-up
operation results in one copy of the object.". Does this means that each
access (even read) to an element of my matrix would imply a new dict
copy?
			


		Yes. While ets outperforms all other alternatives for
random access for sets
		of small objects (at least sets of size > 20), it
doesn't necessarily do so for
		large objects. 
		
		But if a dict-of-dict structure works, you might want to
consider just using an 
		ordset ets table. If you have a tuple as the key in an
ordset table, ets:select(),
		using a pattern where the first part of the key is
bound, will be very efficient.
		
		You can also create ets tables where the tuples stored
have only one element - 
		the key. Mnesia doesn't let you do that, but ets does.
		
		BR,
		Ulf W
		


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070416/a8c31157/attachment.htm>