[erlang-questions] Dangers of generating a large erlang module

[Michael Truog]

Depending on what all the pattern matching looked like, you could use strings (lists of integers) instead of binaries, with a trie (https://github.com/okeuday/trie) using patterns.  So the function you mentioned would look like:
trie:store("trigger*", ..., Trie)

Later, you would use trie:find_match/2 to get a match, and trie:pattern_parse/2 to get any pattern matched values (in this case, whatever comes after "trigger").  You could use ct_expand from https://github.com/uwiger/parse_trans to expand it at compile time, but it may still consume too much memory.  Just an idea, it still seems nice being able to have a static lookup in a module that goes through a regular compilation.


On 09/29/2013 03:50 AM, Ivan Uemlianin wrote:
> Dear Anthony
>
> Thanks for your comment.
>
> Yes, that's exactly what the generated module is doing.  The generated module has a single function with many clauses like this:
>
>     f(<<"trigger", Rest/binary) -> ...
>
> This is why (as far as I can work out) the generated code has to be so big.
>
> I prefer the idea of generating and loading code to, say, updating a database table, because it seems faster and less likely to lead to bottlenecks.
>
> Best wishes
>
> Ivan
>
>
> On 29/09/2013 11:38, Anthony Ramine wrote:
>> Hello Ivan,
>>
>> Out of curiosity, what does it look like?
>>
>> Pattern matching on literal values in Erlang is done with a binary search over the sorted list of patterns, I am not sure this would play well with your use case even if the compilation didn't bring the VM down.
>>
>> Regards,
>>
>> Le 29 sept. 2013 à 11:29, Ivan Uemlianin a écrit :
>>
>>> All goes well on small test files, but the files I want to use IRL are relatively large --- around 120,000 lines.
>>
>