[erlang-questions] Dangers of generating a large erlang module

[Erik Søe Sørensen]

A thing which I discovered recently (in connection with mochiglobal) is
that compiling code containing large binaries, or large amounts of
binaries,  is quite memory-intensive. As I recall it, the numbers were ~64
bytes of RAM per byte in a binary metal; twice as much if on a 64 bit
emulator.
Which means that if you want to compile modules containing (in sum)
multimegabyte binaries, doing so from Erlang source or from full Erlang AST
is a no-go.  Iirc, it is feasible if starting from Core Erlang.
/Erik
Den 29/09/2013 12.50 skrev "Ivan Uemlianin" <ivan@REDACTED>:

> Dear Anthony
>
> Thanks for your comment.
>
> Yes, that's exactly what the generated module is doing.  The generated
> module has a single function with many clauses like this:
>
>     f(<<"trigger", Rest/binary) -> ...
>
> This is why (as far as I can work out) the generated code has to be so big.
>
> I prefer the idea of generating and loading code to, say, updating a
> database table, because it seems faster and less likely to lead to
> bottlenecks.
>
> Best wishes
>
> Ivan
>
>
> On 29/09/2013 11:38, Anthony Ramine wrote:
>
>> Hello Ivan,
>>
>> Out of curiosity, what does it look like?
>>
>> Pattern matching on literal values in Erlang is done with a binary search
>> over the sorted list of patterns, I am not sure this would play well with
>> your use case even if the compilation didn't bring the VM down.
>>
>> Regards,
>>
>> Le 29 sept. 2013 à 11:29, Ivan Uemlianin a écrit :
>>
>>  All goes well on small test files, but the files I want to use IRL are
>>> relatively large --- around 120,000 lines.
>>>
>>
>>
> --
> ==============================**==============================
> Ivan A. Uemlianin PhD
> Llaisdy
> Speech Technology Research and Development
>
>                     ivan@REDACTED
>                      www.llaisdy.com
>                          llaisdy.wordpress.com
>               github.com/llaisdy
>                      www.linkedin.com/in/**ivanuemlianin<http://www.linkedin.com/in/ivanuemlianin>
>
>                         festina lente
> ==============================**==============================
> ______________________________**_________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130929/8a9f3fb0/attachment.htm>