[erlang-questions] Re: Shared/Hybrid Heap

Fri Nov 5 08:05:29 CET 2010

If you're interested ... on my MacBook pro 2.53GHz Intel Core 2 Duo, I get the following numbers:

[HiPE, erts-5.8]
$ time erl -noshell -run  chameneosredux main 6000000

real 1m16.766s
user 1m34.237s
sys 0m44.391s

CPU load is ~ [95%, 95%]
top reporting ~6MB real memory

For some reason, BEAM (sans HiPE) is slightly faster

real 1m9.116s
user 1m25.769s
sys 0m35.669s

Now erjang, ...

[Erjang, erts-5.8]
$ time ej +S 1 -noshell -run  chameneosredux main 6000000

real 0m46.366s
user 0m47.458s
sys 0m0.606s

CPU load is ~[50%, 50%]
top reporting ~75MB real memory
java invoked using -server -Xmx15mb          [15 mb heap]

Increasing the heap size does not make Erjang go faster/slower, but ~10MB seems to be the working set size of the heap.  I.e. setting the total heap size below that will put significant pressure on the GC.

Erjang on 64-bit Java and a double the heap runs marginally faster.

For reference, the plain "java -server version" http://shootout.alioth.debian.org/u32q/benchmark.php?test=chameneosredux&lang=java
runs like this on my machine

real 0m4.900s
user 0m8.460s
sys 0m0.098s

So my machine runs ~ 1.6 x faster than the one used in the shootout.

This should be very easy to try on an Azul box; but right now I think the bottleneck will be that my schedulers will end up competing for work in an dumb way.  Running erjang +S 2 (i.e. one scheduler per core) makes erjang run at-par with BEAM/HiPE.

Interestingly, the profile for the running Erjang looks like this, and for some reason the critical function EProc.execute is never JIT'ed.  Will have to figure out why that is and there should be a potential huge win.  [enabling the Java profiler slows it down a bit, which is why this is a profile of ~52 seconds :-]

Flat profile of 51.83 secs (4452 total ticks): Thread-1

  Interpreted + native   Method
 78.4%  3399  +     0    erjang.EProc.execute
  1.4%     7  +    52    java.lang.Class.getDeclaredConstructors0
  0.5%     0  +    23    java.util.zip.Inflater.inflateBytes
  0.4%    18  +     0    java.lang.ClassLoader.defineClass1
  0.3%     0  +    13    java.lang.Class.forName0
  0.1%     0  +     5    java.util.zip.ZipFile.read
  0.1%     5  +     0    erjang.m.erlang.ErlBif.apply
  0.1%     2  +     3    java.security.AccessController.doPrivileged
  0.1%     5  +     0    kilim.Fiber.togglePause

     Compiled + native   Method
  2.9%    51  +    74    erjang.m.chameneosredux.chameneosredux.chameneos__4
  2.3%    83  +    17    kilim.Mailbox.untilHasMessages
  2.0%    67  +    19    erjang.m.chameneosredux.chameneosredux.broker__1
  1.6%    71  +     0    erjang.EHandle.send
  1.6%    44  +    25    erjang.ERT.wait
  1.6%    68  +     0    kilim.WorkerThread.run
  0.8%    36  +     0    erjang.ERT.send
  0.4%    17  +     0    erjang.m.erlang.ErlBif.apply
  0.2%    10  +     0    erjang.m.chameneosredux.chameneosredux$FN_chameneos__4.go
  0.2%     9  +     0    erjang.m.chameneosredux.chameneosredux.main__1

Kresten

On Nov 4, 2010, at 23:17 , Tony Arcieri wrote:

Hi Kresten,

I'd be very interested to know how Chameneos performs under Erjang on an Azul system, although a comparison with BEAM might prove difficult:

http://shootout.alioth.debian.org/u32q/performance.php?test=chameneosredux

<http://shootout.alioth.debian.org/u32q/performance.php?test=chameneosredux>It seems like the type of problem that could really benefit from Erjang and its shared heap.

On Thu, Nov 4, 2010 at 12:29 PM, Kresten Krab Thorup <krab@REDACTED<mailto:krab@REDACTED>> wrote:
Yes, it would be interesting to know how erjang behaves in such an environment. Have been talking to Gil Tene (Azul CTO) here at QCon about doing that; but we need some system to test that is sufficiently complex, runs on erjang, and who's behavior is known so we can make relevant evaluation. Any takers?  I can help connect people and make this happen.

Kresten

On 04/11/2010, at 09.39, "Tony Arcieri" <tony.arcieri@REDACTED<mailto:tony.arcieri@REDACTED>> wrote:

> On Wed, Nov 3, 2010 at 6:50 PM, Robert Virding <
> robert.virding@REDACTED<mailto:robert.virding@REDACTED>> wrote:
>
>> 1. Shared/hybrid heaps are wonderful BUT:
>> - They are more difficult to implement in a real-time environment. Yes,
>> this is a known problem with solutions but the benefits are less than you
>> would expect. Real-time gc costs.
>> - They don't port easily to multi-core systems as you suddenly start
>> needing locks and stuff everywhere to control access or have complex schemes
>> to get around this. Which copying doesn't.
>>
>
> There's been a considerable amount of research on this matter on the Java
> platform. The Azul Java platform addresses both of these concerns, with a
> pauseless GC designed for massively multi-core systems (54 cores per CPU):
>
> http://www.azulsystems.com/products/vega/overview
>
> I'd be curious how well Erjang would work on such a system, particularly on
> the types of shared state concurrency problems that don't work well on BEAM,
> like Chameneos.
>
> --
> Tony Arcieri
> Medioh! A Kudelski Brand

--
Tony Arcieri
Medioh! A Kudelski Brand

Kresten Krab Thorup, CTO, Trifork