[erlang-questions] Dangers of generating a large erlang module

Ivan uemlianin ivan@REDACTED
Sun Sep 29 19:20:03 CEST 2013

Thanks! I think I'll try and head in that direction. I've had a few goes at other methods (db lookup etc) and they're much slower than this "dynamic hardcoding"). I'll sniff around for Core Erlang tutorials.

Best wishes


festina lente

On 29 Sep 2013, at 17:48, Erik Søe Sørensen <eriksoe@REDACTED> wrote:

> A thing which I discovered recently (in connection with mochiglobal) is that compiling code containing large binaries, or large amounts of binaries,  is quite memory-intensive. As I recall it, the numbers were ~64 bytes of RAM per byte in a binary metal; twice as much if on a 64 bit emulator.
> Which means that if you want to compile modules containing (in sum) multimegabyte binaries, doing so from Erlang source or from full Erlang AST is a no-go.  Iirc, it is feasible if starting from Core Erlang.
> /Erik
> Den 29/09/2013 12.50 skrev "Ivan Uemlianin" <ivan@REDACTED>:
>> Dear Anthony
>> Thanks for your comment.
>> Yes, that's exactly what the generated module is doing.  The generated module has a single function with many clauses like this:
>>     f(<<"trigger", Rest/binary) -> ...
>> This is why (as far as I can work out) the generated code has to be so big.
>> I prefer the idea of generating and loading code to, say, updating a database table, because it seems faster and less likely to lead to bottlenecks.
>> Best wishes
>> Ivan
>> On 29/09/2013 11:38, Anthony Ramine wrote:
>>> Hello Ivan,
>>> Out of curiosity, what does it look like?
>>> Pattern matching on literal values in Erlang is done with a binary search over the sorted list of patterns, I am not sure this would play well with your use case even if the compilation didn't bring the VM down.
>>> Regards,
>>> Le 29 sept. 2013 à 11:29, Ivan Uemlianin a écrit :
>>>> All goes well on small test files, but the files I want to use IRL are relatively large --- around 120,000 lines.
>> -- 
>> ============================================================
>> Ivan A. Uemlianin PhD
>> Llaisdy
>> Speech Technology Research and Development
>>                     ivan@REDACTED
>>                      www.llaisdy.com
>>                          llaisdy.wordpress.com
>>               github.com/llaisdy
>>                      www.linkedin.com/in/ivanuemlianin
>>                         festina lente
>> ============================================================
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130929/a7c5bece/attachment.htm>

More information about the erlang-questions mailing list