R14B, ETS, read_concurrency
Thu Sep 16 14:13:54 CEST 2010
Ryan Zezeski wrote:
> After glancing at the R14B release notes I noticed the SMP related
> improvements that were made. I specifically find the new read_concurrency
> ETS opt interesting. I currently have an app that performs roughly 480
> million lookups (in worst case) on multiple ETS tables, with some tables
> having up to 10 million entries. I perform these lookups in parallel using
> 8 processes (on a 16 core machine). This is the longest running part of my
> program with a runtime of > 3min.
> My point--I am doing a _huge_ amount of concurrent reads and every
> microsecond counts. I am curious if anyone could throw a minimum % increase
> of lookups per second I might expect?
What the effect will be on your application is impossible to guess. This
depends on how writes are done in the tables using the read_concurrency
option (if you have lots of write locked operations too you may even get
a performance degradation), what else your processes do (also other
processes not accessing the tables), what hardware you use, etc...
In order to give some figures, though. When doing lots of ets lookups
(only lookups) of small records on a public table from lots (hundreds)
of processes I get a speedup of about 5 using read_concurrency compared
to not using it on a dual quad-core machine we got, and about 2.5 on a
quad-core machine we got. The only thing these processes do are repeated
ets lookups. You should, however, not expect figures like these for
normal apps, since they typically do other stuff than ets lookup too.
The only way to know what effect it will have on your app is to try it out.
Rickard Green, Erlang/OTP, Ericsson AB.
More information about the erlang-questions