HiPE to be removed in OTP 24?

[Richard O'Keefe]

>From watching the BEAMJIT video, they *tried* building on top of clang, and
that turned out to be a mistake.  Then they *tried* building on top of
LLVM, and that turned out to be a mistake too.  In fact the detours through
Clang and LLVM cost so much time that the whole BEAMJIT idea was very
nearly shelved sine die.

Just because clang & co are getting optimisers &c does not mean that they
are getting optimisers that pay off *for Erlang*.

By the way, I am sick of the way people credit Java with the idea of JITs.
The idea goes back to load-and-go systems, via Lisp, Brown's invention of
"throwaway compiling" for a BASIC system, to Smalltalk (Deutsch &
Schiffman), and Self.  Indeed, one issue raised in the video could have
been answered by considering Brown's throw-away compiling and the
application of that idea in early commercial Smalltalk.

On Sat, 20 Jun 2020 at 12:35, Felix Gallo <felixgallo@REDACTED> wrote:

> Given the highly constrained resources of the OTP team, I would generally
> expect them to go the other way, and try to hitch their wagons to large
> systems being built or maintained by others.  The cost of building a de
> novo high performing JIT and then committing to its ongoing maintenance and
> development doesn't look, from the outside, like the kind of investment
> that the OTP team has traditionally made.  Even relatively fundamental
> subsystems tend to age out for lack of curation.  Is there a new source of
> funding or staffing which gives the team confidence that they'll be able to
> make that commitment?
>
> To ROK's point more specifically -- many new architectures are emerging
> these days and it's not clear that x86 is the long term winner -- various
> flavors of ARM, at a minimum, look to be contenders in the near future, at
> both the consumer and industrial levels.  Clang + co are clearly going to
> keep getting optimizers and JITs for those environments through the work of
> thousands of well-incentivized contributors.  What does the BEAMJIT vector
> look like by comparison?
>
> F.
>
> On Fri, Jun 19, 2020 at 3:42 PM Tristan Sloughter <t@REDACTED> wrote:
>
>> The JIT will support tracing like normal. So even if it was on-par or
>> even a little slower than HiPE I think that is reason enough alone to
>> switch -- and reason enough to have been attempting at alternatives all
>> these years.
>>
>> JIT could be a solution that makes sense in general usage while HIPE
>> because of its limitations (even before recent OTP changes that diverged
>> from HIPE even more so) meant it was a niche tool. Very useful to those
>> problems it fit, but still niche.
>>
>> Tristan
>>
>> On Fri, Jun 19, 2020, at 10:45, Grzegorz Junka wrote:
>>
>> Hi Kenneth,
>>
>> Thanks to Richard's link I was able to see the presentation about JIT and
>> it looks like the team was putting a lot of effort since as early as 2012
>> into JIT. But on none of the benchmarks JIT was actually better than HiPE.
>>
>> I am interested, in what way JIT is better than HiPE that it's worth
>> putting the effort into developing something new instead of fixing HiPE?
>>
>> Regards
>>
>> Grzegorz
>>
>>
>> On 18/06/2020 08:35, Kenneth Lundin wrote:
>>
>>
>> HiPE is the runtime and compiler support for native code generation of
>> Erlang modules that some of you might have tried, it is part of the OTP
>> repository today.
>>
>> The OTP team is planning to remove HiPE in the OTP 24 release for the
>> following reasons:
>>
>>    - we plan to introduce a new way of executing Erlang, the "JIT"
>>    described by Lukas Larsson at Code Beam V
>>    - since OTP 22, HiPE is not fully functional (does not handle all
>>    beam instructions and combinations)
>>    - there is no use of HiPE among our primary customers. We actually
>>    don't know where HiPE is used except for speeding up Dialyzer which we have
>>    another solution for.
>>    - The current support for HiPE in the code is a blocker or creates
>>    extra work in our new development.
>>
>> In order to not remove HiPE in OTP 24, we really soon need maintainers
>> committing (long term) to keep HiPE in shape and up to date with the rest
>> of OTP.
>> /Kenneth Erlang/OTP, Ericsson
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20200620/c8c265c7/attachment.htm>