[erlang-questions] Re: Erjang

[Thomas Lindgren]

----- Original Message ----
> From: Tony Arcieri <tony@REDACTED>
> To: Johann Höchtl <johann.hoechtl@REDACTED>
> Cc: erlang-questions@REDACTED
> Sent: Tue, December 8, 2009 6:58:44 PM
> Subject: Re: [erlang-questions] Re: Erjang
> 
> On Tue, Dec 8, 2009 at 10:47 AM, Johann Höchtl wrote:
> 
> > Given the trmendous work which has been done on Luajit2 and the vast
> > improvements speedwise, I wonder I sthg. a least mildly comparabel
> > could be done to the Beam VM. Is some work going on for HIPE or the VM
> > in general?
> >
> 
> HiPE is nowhere close to the JVM's HotSpot in terms of the types of
> optimizations in can perform.
> 
> Among other things, the JVM's HotSpot can deoptimize.  HiPE cannot, and for
> this reason HiPE cannot inline across modules, as a code change would
> require the inliner to deoptimize whenever code change occurs.
> Deoptimization also allows HotSpot to try aggressive optimizations which may
> actually lead to performance regressions or no noticeable improvements, in
> which case it can deoptimize and try again.

(Note that it's been known since 1996 how to inline and optimize across modules in Erlang, with some preliminary experimental results in 2001 and 2003 even. See http://www.erlang.org/pipermail/erlang-questions/2008-November/039944.html for pointers.)

As far as I can tell, the community currently seems pretty satisfied with performance, to the point of not even using Hipe a lot. The current compiler architecture seems to work best for small or medium-sized programs, but loses steam on big programs. With that said, I think there is still plenty of scope to improve performance. Languages like OCaml and Haskell seem to be running (rough estimate) 2x-10x faster on some benchmarks. A compiler like Stalin (for Scheme) also does extraordinarily well. So it's not like we've maxed out the hardware, and closing the gap seems worthwhile.

I for one thus think it would be very interesting to see more work on optimization. The situation is pretty bright for the interested. There are lots of open source real apps to use as benchmarks these days, if so desired. (Downside: can also provide massive distractions.) Hipe is very solid at this point, it's easy to use it for comparisons, it's easy to examine the code generated, and the system includes a lot of work on machine code generation, linking, converting beam ops into low-level code, and so on, all of which quite possibly can be reused to quickly do a luajit competitor, or the infrastructure for one. (You may also need to do some further runtime system hacking.)

Good luck!

Best,
Thomas