HiPE to be removed in OTP 24?

[Rich Neswold]

Yes, but it was a very specific case in how we where building a binary. And
this was OTP 18.

We weren't using HiPE at the time and our protocol generator was producing
acceptable code. Then a colleague and I decided to test the generator with
larger arrays so we marshaled 2048 floating point values (among other
things in the protocol message.) It took about 80 ms to create the binary!
One thing we noticed was we didn't have +native when compiling. Once we did
that, we could build the messages in under 1 ms. So, for this very specific
case, it was ~100x faster.

HiPE wasn't giving us 100x across the board. I think, in OTP18, our code
stumbled across a use of binaries that HiPE handled much better than the
byte-code. As we upgraded to new OTP releases, we didn't bother to see if
we could turn off +native.


On Fri, Jun 19, 2020, 10:34 PM Max Lapshin <max.lapshin@REDACTED> wrote:

> You really get 100x speedup with hipe?
>
> On Fri, Jun 19, 2020 at 8:32 PM Rich Neswold <rich.neswold@REDACTED>
> wrote:
> >
> > We are not a “primary customer” but HiPE is essential at Fermilab. We
> have an in-house, protocol code generator that aggressively builds binaries
> using bitstream comprehensions. We also adjusted our generator to follow
> hints given from +bin_opt_info. The last time we measured performance
> between byte-code and HiPE, we found the native code ran 100x faster making
> Erlang a viable option for our systems.
> >
> > We’re currently at OTP 21 and have no problem staying there until we
> feel the JIT performs similarly and supports ARM. I’m building OTP 23
> without HiPE on a test system to see if the binary manipulation
> instructions improved enough that this isn’t an issue.
> >
> > Just thought I’d let you know that HiPE does have its users.
> >
> >
> > On Thu, Jun 18, 2020, 3:35 AM Kenneth Lundin <kenneth@REDACTED> wrote:
> >>
> >> HiPE is the runtime and compiler support for native code generation of
> Erlang modules that some of you might have tried, it is part of the OTP
> repository today.
> >>
> >> The OTP team is planning to remove HiPE in the OTP 24 release for the
> following reasons:
> >>
> >> we plan to introduce a new way of executing Erlang, the "JIT" described
> by Lukas Larsson at Code Beam V
> >> since OTP 22, HiPE is not fully functional (does not handle all beam
> instructions and combinations)
> >> there is no use of HiPE among our primary customers. We actually don't
> know where HiPE is used except for speeding up Dialyzer which we have
> another solution for.
> >> The current support for HiPE in the code is a blocker or creates extra
> work in our new development.
> >>
> >> In order to not remove HiPE in OTP 24, we really soon need maintainers
> committing (long term) to keep HiPE in shape and up to date with the rest
> of OTP.
> >>
> >> /Kenneth Erlang/OTP, Ericsson
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20200620/e4665cb5/attachment.htm>