[erlang-questions] performance drop in R15A?

[Nico Kruber]

Tonight, I ran the tests with an erlang checkout from around 16:00 yesterday
(erlang-otp-OTP_R15A-126-g0290bed.tar.gz).
This includes the fix Björn-Egil was referring to,
i.e. 'rickard/pix-mutex/OTP-9723'.


Following are the results or our "make bench" of Scalaris:

** transactions/s , i.e. "wall clock throughput  : xxx" **
R14B04: ~950, R15A (ge21ff9b): ~800, R15A (g0290bed): ~635

** latency, i.e. "wall clock avg. latency: xxx ms" **:
R14B04: ~42ms, R15A (ge21ff9b): ~50ms, R15A (g0290bed): ~60ms


Note: This "make bench" command sets up a Scalaris ring consisting of 4 nodes, 
each in a single VM. They all communicate through tcp.


Additionally, we also measure our performance through our Java-API which 
accesses Erlang via JInterface using
"SCALARIS_BENCHMARKS="100 6 9 15 18" make java-bench"

** number of transactions per second for "Benchmark incrementing an 
integer key (read+write):"
R14B04: ~310, R15A (ge21ff9b): ~200, R15A (g0290bed): ~140


I saw some new bug fixes for the scheduler in the recent commits, but with the 
checkout from 6971b96 (just now), the results are still the same as with 
g0290bed :(



Regards,
Nico


On Wednesday 30 November 2011 17:05:29 Nico Kruber wrote:
> thank you
> We'll try again tomorrow
> 
> Nico
> 
> On Wednesday 30 November 2011 16:30:05 Lukas Larsson wrote:
> > Hello!
> > 
> > I was able to verify you findings on my machine for R14B04 and R15A. The
> > problem is however fixed (I even get a 15% throughput improvement on
> > your
> > bench) in our latest internal master which will be pushed to github
> > later
> > this evening. So if you could pull the latest from github tomorrow and
> > verify my findings that would be great!
> > 
> > Lukas
> > 
> > On Wed, Nov 30, 2011 at 11:36 AM, Nico Kruber <kruber@REDACTED> wrote:
> > > Is it possible, that some debug option is on by default in the R15A
> > > tagged git
> > > checkout?
> > > We build the source using the following commands (on a x86_64 system
> > > with 8 cores):
> > > 
> > > ./otp_build autoconf
> > > ./configure --prefix=/home/buildbot/apps/erlang-R15A
> > > make
> > > make install
> > > 
> > > and then compiled our application
> > > (http://code.google.com/p/scalaris/)
> > > using
> > > R15A.
> > > 
> > > With our internal benchmarks ("make bench") showed a drop of around
> > > 15%
> > > in performance compared to R14B04. We were actually hoping for an
> > > increased performance reading about the memory allocation
> > > improvements...
> > > 
> > > Is there some switch (during configuration or build) influencing the
> > > performance?
> > > 
> > > output of R14B04 erl:
> > > ------------------------------------------------------------
> > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:8:8] [rq:8]
> > > [async-
> > > threads:0] [hipe] [kernel-poll:false]
> > > 
> > > Eshell V5.8.5  (abort with ^G)
> > > 1>
> > > ------------------------------------------------------------
> > > 
> > > output of R15A erl:
> > > ------------------------------------------------------------
> > > Erlang R15A (erts-5.9.pre) [source] [64-bit] [smp:8:8] [rq:8]
> > > [async-
> > > threads:0] [hipe] [kernel-poll:false]
> > > 
> > > Eshell V5.9.pre  (abort with ^G)
> > > 1>
> > > ------------------------------------------------------------
> > > 
> > > 
> > > Regards
> > > Nico Kruber
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111202/81e1fa89/attachment.bin>