ets memory usage

[Colm Dougan]

Gah - my email client sent this before I had finished typing it.  But
my question was really just: do my numbers seem reasonable and is
there a way to be less memory intensive.  BTW - my reference to a
"30GB cache" was in relation to a block store we have in our app which
uses 4k blocks.

Thanks,
Colm

On Thu, Aug 20, 2009 at 5:33 PM, Colm Dougan<colm.dougan@REDACTED> wrote:
> Hi,
>
> A question about ets memory usage :
>
> Eshell V5.6.5  (abort with ^G)
> 1> Tab = ets:new(foo, [set]).
> 16397
> 2> lists:foreach(fun(I) -> ets:insert(Tab, {I, 0}) end, lists:seq(1, 1000000)).
> ok
> 3> erlang:memory(ets).
> 48684012
>
> So that looks like more than 40 bytes per entry.  Pretty expensive, no?
>
> How about when the key is an md5sum :
>
> Eshell V5.6.5  (abort with ^G)
> 1> Tab = ets:new(foo, [set]).
> 16397
> 2>  lists:foreach(fun(I) -> ets:insert(Tab, {erlang:md5(<<I:32>>), 0})
> end, lists:seq(1, 1000000)).
> ok
> 3> erlang:memory(ets).
> 72684940
>
> So that is an additional 32 bytes per entry for an md5sum key instead
> of a (4 byte?) integer key.
>
> For a 30GB cache we would need about 7 million index entries which
> would minimally take up
>