[erlang-questions] Generating Core Erlang -- Re: Dangers of generating a large erlang module
Anthony Ramine
n.oxyde@REDACTED
Mon Sep 30 10:08:59 CEST 2013
Hello Erik,
Use +time to measure which passes are taking the most time. IIRC the slow pass with regard to modules with lot of binaries is either a Core Erlang optimization pass or the Core to Kernel v3_kernel one. In both cases, targeting Core directly won't fix the problem.
Regards,
Le 30 sept. 2013 à 07:31, Erik Søe Sørensen a écrit :
> Yes, indeed; considering that the goal is reduction of memory use, however, what we want is to avoid intermediate representations which explode in size on binaries. I don't know anything about the from_core parser, but the string-as-list representation of a binary literal is in itself a factor 32/64 blowup. :-/
> (Btw: I was wrong about introductions to Core Erlang; threre is one. Writing on the phone has its fact-checking disadvantages...)
>
> Den 29/09/2013 23.01 skrev "Tom Murphy" <amindfv@REDACTED>:
> There's also just:
>
> erlc +to_core foo.erl
> erlc +from_core foo.core
>
> Tom Murphy
>
>
> On Sun, Sep 29, 2013 at 11:59 AM, Loïc Hoguin <essen@REDACTED> wrote:
> On 09/29/2013 08:24 PM, Ivan uemlianin wrote:
> That's what I've just done :D Core Erlang looks very verbose but quite
> regular & probably not difficult to generate.
>
> My questions now are:
> - are there any libraries "out there" for generating Core Erlang, or do
> we all roll our own?
>
> Look at the cerl module. It's just a matter of generating the proper structure using those functions.
>
>
> - how would one use compile:file or compile:forms with core erlang? I
> haven't been able to find any documentation (haven't read Richard
> Carlsson's Introduction paper yet).
>
> Do read it. But to answer that question, compile:forms with from_core option.
>
> Many thanks
>
> Ivan
>
> --
> festina lente
>
>
> On 29 Sep 2013, at 18:36, Erik Søe Sørensen <eriksoe@REDACTED
> <mailto:eriksoe@REDACTED>> wrote:
>
> Core Erlang is an intermediate representation in the Erlang compiler
> - but also (afaik) a fairly well-defined/public one and one that is
> stable.
> I don't think you'll find much in the vein of tutorials. Try getting
> erlc to output the intermediate format, though, for a small program
> similar to what you'll be using it for.
>
> Den 29/09/2013 19.20 skrev "Ivan uemlianin" <ivan@REDACTED
> <mailto:ivan@REDACTED>>:
>
>
> Thanks! I think I'll try and head in that direction. I've had a
> few goes at other methods (db lookup etc) and they're much slower
> than this "dynamic hardcoding"). I'll sniff around for Core Erlang
> tutorials.
>
> Best wishes
>
> Ivan
>
>
> --
> festina lente
>
>
> On 29 Sep 2013, at 17:48, Erik Søe Sørensen <eriksoe@REDACTED
> <mailto:eriksoe@REDACTED>> wrote:
>
> A thing which I discovered recently (in connection with
> mochiglobal) is that compiling code containing large binaries, or
> large amounts of binaries, is quite memory-intensive. As I
> recall it, the numbers were ~64 bytes of RAM per byte in a binary
> metal; twice as much if on a 64 bit emulator.
> Which means that if you want to compile modules containing (in
> sum) multimegabyte binaries, doing so from Erlang source or from
> full Erlang AST is a no-go. Iirc, it is feasible if starting
> from Core Erlang.
> /Erik
>
> Den 29/09/2013 12.50 skrev "Ivan Uemlianin" <ivan@REDACTED
> <mailto:ivan@REDACTED>>:
>
>
> Dear Anthony
>
> Thanks for your comment.
>
> Yes, that's exactly what the generated module is doing. The
> generated module has a single function with many clauses like
> this:
>
> f(<<"trigger", Rest/binary) -> ...
>
> This is why (as far as I can work out) the generated code has
> to be so big.
>
> I prefer the idea of generating and loading code to, say,
> updating a database table, because it seems faster and less
> likely to lead to bottlenecks.
>
> Best wishes
>
> Ivan
>
>
> On 29/09/2013 11:38, Anthony Ramine wrote:
>
> Hello Ivan,
>
> Out of curiosity, what does it look like?
>
> Pattern matching on literal values in Erlang is done with
> a binary search over the sorted list of patterns, I am
> not sure this would play well with your use case even if
> the compilation didn't bring the VM down.
>
> Regards,
>
> Le 29 sept. 2013 à 11:29, Ivan Uemlianin a écrit :
>
> All goes well on small test files, but the files I
> want to use IRL are relatively large --- around
> 120,000 lines.
>
>
>
> --
> ==============================__==============================
>
> Ivan A. Uemlianin PhD
> Llaisdy
> Speech Technology Research and Development
>
> ivan@REDACTED <mailto:ivan@REDACTED>
> www.llaisdy.com <http://www.llaisdy.com>
> llaisdy.wordpress.com <http://llaisdy.wordpress.com>
> github.com/llaisdy <http://github.com/llaisdy>
> www.linkedin.com/in/__ivanuemlianin
> <http://www.linkedin.com/in/ivanuemlianin>
>
> festina lente
> ==============================__==============================
> _________________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
> http://erlang.org/mailman/__listinfo/erlang-questions
> <http://erlang.org/mailman/listinfo/erlang-questions>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
> --
> Loïc Hoguin
> Erlang Cowboy
> Nine Nines
> http://ninenines.eu
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
--
Anthony Ramine
More information about the erlang-questions
mailing list