[erlang-questions] Re: Shared/Hybrid Heap
Kresten Krab Thorup
Fri Nov 5 08:05:29 CET 2010
If you're interested ... on my MacBook pro 2.53GHz Intel Core 2 Duo, I get the following numbers:
$ time erl -noshell -run chameneosredux main 6000000
CPU load is ~ [95%, 95%]
top reporting ~6MB real memory
For some reason, BEAM (sans HiPE) is slightly faster
Now erjang, ...
$ time ej +S 1 -noshell -run chameneosredux main 6000000
CPU load is ~[50%, 50%]
top reporting ~75MB real memory
java invoked using -server -Xmx15mb [15 mb heap]
Increasing the heap size does not make Erjang go faster/slower, but ~10MB seems to be the working set size of the heap. I.e. setting the total heap size below that will put significant pressure on the GC.
Erjang on 64-bit Java and a double the heap runs marginally faster.
For reference, the plain "java -server version" http://shootout.alioth.debian.org/u32q/benchmark.php?test=chameneosredux&lang=java
runs like this on my machine
So my machine runs ~ 1.6 x faster than the one used in the shootout.
This should be very easy to try on an Azul box; but right now I think the bottleneck will be that my schedulers will end up competing for work in an dumb way. Running erjang +S 2 (i.e. one scheduler per core) makes erjang run at-par with BEAM/HiPE.
Interestingly, the profile for the running Erjang looks like this, and for some reason the critical function EProc.execute is never JIT'ed. Will have to figure out why that is and there should be a potential huge win. [enabling the Java profiler slows it down a bit, which is why this is a profile of ~52 seconds :-]
Flat profile of 51.83 secs (4452 total ticks): Thread-1
Interpreted + native Method
78.4% 3399 + 0 erjang.EProc.execute
1.4% 7 + 52 java.lang.Class.getDeclaredConstructors0
0.5% 0 + 23 java.util.zip.Inflater.inflateBytes
0.4% 18 + 0 java.lang.ClassLoader.defineClass1
0.3% 0 + 13 java.lang.Class.forName0
0.1% 0 + 5 java.util.zip.ZipFile.read
0.1% 5 + 0 erjang.m.erlang.ErlBif.apply
0.1% 2 + 3 java.security.AccessController.doPrivileged
0.1% 5 + 0 kilim.Fiber.togglePause
Compiled + native Method
2.9% 51 + 74 erjang.m.chameneosredux.chameneosredux.chameneos__4
2.3% 83 + 17 kilim.Mailbox.untilHasMessages
2.0% 67 + 19 erjang.m.chameneosredux.chameneosredux.broker__1
1.6% 71 + 0 erjang.EHandle.send
1.6% 44 + 25 erjang.ERT.wait
1.6% 68 + 0 kilim.WorkerThread.run
0.8% 36 + 0 erjang.ERT.send
0.4% 17 + 0 erjang.m.erlang.ErlBif.apply
0.2% 10 + 0 erjang.m.chameneosredux.chameneosredux$FN_chameneos__4.go
0.2% 9 + 0 erjang.m.chameneosredux.chameneosredux.main__1
On Nov 4, 2010, at 23:17 , Tony Arcieri wrote:
I'd be very interested to know how Chameneos performs under Erjang on an Azul system, although a comparison with BEAM might prove difficult:
<http://shootout.alioth.debian.org/u32q/performance.php?test=chameneosredux>It seems like the type of problem that could really benefit from Erjang and its shared heap.
On Thu, Nov 4, 2010 at 12:29 PM, Kresten Krab Thorup <krab@REDACTED<mailto:krab@REDACTED>> wrote:
Yes, it would be interesting to know how erjang behaves in such an environment. Have been talking to Gil Tene (Azul CTO) here at QCon about doing that; but we need some system to test that is sufficiently complex, runs on erjang, and who's behavior is known so we can make relevant evaluation. Any takers? I can help connect people and make this happen.
On 04/11/2010, at 09.39, "Tony Arcieri" <tony.arcieri@REDACTED<mailto:tony.arcieri@REDACTED>> wrote:
> On Wed, Nov 3, 2010 at 6:50 PM, Robert Virding <
> robert.virding@REDACTED<mailto:robert.virding@REDACTED>> wrote:
>> 1. Shared/hybrid heaps are wonderful BUT:
>> - They are more difficult to implement in a real-time environment. Yes,
>> this is a known problem with solutions but the benefits are less than you
>> would expect. Real-time gc costs.
>> - They don't port easily to multi-core systems as you suddenly start
>> needing locks and stuff everywhere to control access or have complex schemes
>> to get around this. Which copying doesn't.
> There's been a considerable amount of research on this matter on the Java
> platform. The Azul Java platform addresses both of these concerns, with a
> pauseless GC designed for massively multi-core systems (54 cores per CPU):
> I'd be curious how well Erjang would work on such a system, particularly on
> the types of shared state concurrency problems that don't work well on BEAM,
> like Chameneos.
> Tony Arcieri
> Medioh! A Kudelski Brand
Medioh! A Kudelski Brand
Kresten Krab Thorup, CTO, Trifork
More information about the erlang-questions