[erlang-questions] Faster ets:lookups

[Scott Lystig Fritchie]

Oops, I forgot to add a comment about Tom's speed question.

By design, ETS storage is separate from a process's private heap.
That's generally considered a good thing, because you can put a zillion
bytes into ETS and not worry about ETS data interfering with the garbage
collection of your process's heap (which is presumably now much much
smaller).  But if you do a lot of lookups, and if the data is "big",
then the VM will be spending a lot of time copying that data out of ETS
and into your process's heap.

Your problem sounds like most of the time, ets:lookup/2 is probably
returning [] most of the time (?), so that data copying overhead
shouldn't be a problem for you.  That said, if ets:member/2 works for
you, you should use member instead of ets:lookup/2 when possible.

I haven't explored R13's ETS implementation, so I don't know if there's
any extra overhead in public vs. protected vs. private tables ... but it
should be easy enough to measure.

-Scott