[erlang-questions] performance drop in R15A?

[Nico Kruber]

Is there any news on this?

The performance drop of the recent R15 checkout compared to R14B04 in our 
application is now at _30%_ compared to a drop of "only" 15% in R15A!


Nico


On Friday 02 December 2011 11:26:56 Nico Kruber wrote:
> Tonight, I ran the tests with an erlang checkout from around 16:00 yesterday
> (erlang-otp-OTP_R15A-126-g0290bed.tar.gz).
> This includes the fix Björn-Egil was referring to,
> i.e. 'rickard/pix-mutex/OTP-9723'.
> 
> 
> Following are the results or our "make bench" of Scalaris:
> 
> ** transactions/s , i.e. "wall clock throughput  : xxx" **
> R14B04: ~950, R15A (ge21ff9b): ~800, R15A (g0290bed): ~635
> 
> ** latency, i.e. "wall clock avg. latency: xxx ms" **:
> R14B04: ~42ms, R15A (ge21ff9b): ~50ms, R15A (g0290bed): ~60ms
> 
> 
> Note: This "make bench" command sets up a Scalaris ring consisting of 4
> nodes, each in a single VM. They all communicate through tcp.
> 
> 
> Additionally, we also measure our performance through our Java-API which
> accesses Erlang via JInterface using
> "SCALARIS_BENCHMARKS="100 6 9 15 18" make java-bench"
> 
> ** number of transactions per second for "Benchmark incrementing an
> integer key (read+write):"
> R14B04: ~310, R15A (ge21ff9b): ~200, R15A (g0290bed): ~140
> 
> 
> I saw some new bug fixes for the scheduler in the recent commits, but with
> the checkout from 6971b96 (just now), the results are still the same as
> with g0290bed :(
> 
> 
> 
> Regards,
> Nico
> 
> On Wednesday 30 November 2011 17:05:29 Nico Kruber wrote:
> > thank you
> > We'll try again tomorrow
> > 
> > Nico
> > 
> > On Wednesday 30 November 2011 16:30:05 Lukas Larsson wrote:
> > > Hello!
> > > 
> > > I was able to verify you findings on my machine for R14B04 and R15A.
> > > The problem is however fixed (I even get a 15% throughput
> > > improvement on your
> > > bench) in our latest internal master which will be pushed to github
> > > later
> > > this evening. So if you could pull the latest from github tomorrow
> > > and
> > > verify my findings that would be great!
> > > 
> > > Lukas
> > > 
> > > On Wed, Nov 30, 2011 at 11:36 AM, Nico Kruber <kruber@REDACTED> wrote:
> > > > Is it possible, that some debug option is on by default in the
> > > > R15A
> > > > tagged git
> > > > checkout?
> > > > We build the source using the following commands (on a x86_64
> > > > system
> > > > with 8 cores):
> > > > 
> > > > ./otp_build autoconf
> > > > ./configure --prefix=/home/buildbot/apps/erlang-R15A
> > > > make
> > > > make install
> > > > 
> > > > and then compiled our application
> > > > (http://code.google.com/p/scalaris/)
> > > > using
> > > > R15A.
> > > > 
> > > > With our internal benchmarks ("make bench") showed a drop of
> > > > around
> > > > 15%
> > > > in performance compared to R14B04. We were actually hoping for
> > > > an
> > > > increased performance reading about the memory allocation
> > > > improvements...
> > > > 
> > > > Is there some switch (during configuration or build) influencing
> > > > the
> > > > performance?
> > > > 
> > > > output of R14B04 erl:
> > > > ------------------------------------------------------------
> > > > Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:8:8] [rq:8]
> > > > [async-
> > > > threads:0] [hipe] [kernel-poll:false]
> > > > 
> > > > Eshell V5.8.5  (abort with ^G)
> > > > 1>
> > > > ------------------------------------------------------------
> > > > 
> > > > output of R15A erl:
> > > > ------------------------------------------------------------
> > > > Erlang R15A (erts-5.9.pre) [source] [64-bit] [smp:8:8] [rq:8]
> > > > [async-
> > > > threads:0] [hipe] [kernel-poll:false]
> > > > 
> > > > Eshell V5.9.pre  (abort with ^G)
> > > > 1>
> > > > ------------------------------------------------------------
> > > > 
> > > > 
> > > > Regards
> > > > Nico Kruber
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111205/06fe8b18/attachment.bin>