[erlang-questions] dynamic compilation of funs

[Ulf Wiger]

There are a few problems with dynamically compiling to (anonymous) modules:

- SMP characteristics: code loading, at the moment, is quite disruptive on SMP.
- GC: you have to heep track of when to purge the module, or you have a memory leak

Figuring out a unique module name is left as an exercise to the reader. :)

On the purging. One way to do this is to make use of the resource objects provided by NIFs.

Here is an example from my own use of sqlite3, whose prepare statements must be removed with finalize(), and my particular implementation couldn't be sure that it would always be called:

track_resource(Ref,Handle) ->
    %% spawn a resource monitor, since a lingering prepare statement may lock the
    %% database (I think...)
    Pid = spawn(fun() -> receive finalize ->
                                 sqlite3:finalize(Ref, Handle)
                         end
                end),
    N = resource:notify_when_destroyed(Pid, finalize),
    {Pid, N, Handle}.

    {ok, Handle} = sqlite3:prepare(Ref, SQL),
    SH = track_resource(Ref, Handle),

Basically, I create a wrapper around the object I want to be able to remove, as soon as the caller no longer references it.

If the resource is accessed concurrently, you would need to introduce a counter, to know when it's safe to remove the resource.

The 'resource' module can be found at https://github.com/tonyrog/resource

BR,
Ulf W

On 24 Apr 2012, at 09:51, Martin Dimitrov wrote:

> Hello,
> 
> I am looking at the dynamic_compile module developed by Mats Cronqvist,
> Chris Newcombe and Jacob Vorreuter. It has a function evaluate/1 which
> uses erl_parse to compile the code. I successfully manage to compile and
> use this way a fun. But going through eprof I start to realize that
> erl_parse has quite a lot of overhead and is inefficient for our needs.
> 
> So what are my other options? I can use dynamic_compile to compile the
> functions into a module. But during the execution of the program new
> functions need to be dynamically compile. How can I achieve that? Can I
> add a function to the dynamically compiled module or I need to recompile
> the whole thing again?
> 
> Thank you very much for looking at this.
> 
> Best regards,
> 
> Martin
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com