[erlang-questions] Futures/promises and ability to "call" abstract modules

Tue Nov 20 15:34:51 CET 2012

How does that spawned process get garbage collected?

Cheers,

DBM

> -----Original Message-----
> From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-
> bounces@REDACTED] On Behalf Of Dmitry Belyaev
> Sent: Monday, November 19, 2012 21:48
> To: Gleb Peregud
> Cc: Erlang
> Subject: Re: [erlang-questions] Futures/promises and ability to "call"
> abstract modules
> 
> Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:2:2] [async-threads:0]
> [hipe] [kernel-poll:false]
> 
> Eshell V5.9.1  (abort with ^G)
> 1> F = (fun() -> FM = fun(M, V) -> receive {value, Value} -> M(M, Value);
> {get, P} -> P ! V, M(M, V) end end, P = spawn(fun() -> FM(FM, undefined)
> end), fun() -> P ! {get, self()}, receive V -> V end end end)().
> #Fun<erl_eval.20.82930912>
> 2> F().
> undefined
> 3> element(2,lists:keyfind('P', 1,
> lists:flatten(element(2,erlang:fun_info(F, env))))) ! {value, 10}.
> {value,10}
> 4> F().
> 10
> 5>
> 
> --
> Dmitry Belyaev
> 
> On 19.11.2012, at 18:05, Gleb Peregud wrote:
> 
> > Then I wouldn't be able to set future's value after it being defined.
> > Like in this example:
> >
> > 7> F = future:new().
> > {future,<0.47.0>,#Ref<0.0.0.27235>,undefined}
> > 8> spawn(fun() -> timer:sleep(10000), F:set(42) end).
> > <0.49.0>
> > 9> F:get().
> > 42
> >
> > Without this ability such futures are useless with request-reply
> pattern:
> > 1) Client sends request
> > 2) Server creates a future and sends it back to client
> > 3) Client does it's work with the value of the future (without
> > bothering the fact that value may become available in uncertain
> > future)
> > 4) Server finishes computations and sets future's value
> >
> >
> > On Mon, Nov 19, 2012 at 3:01 PM, Robert Virding
> > <robert.virding@REDACTED> wrote:
> >> Wouldn't it be easier if future:new just returned a fun? Then you could
> do F() without any changes?
> >>
> >> Robert
> >>
> >> ----- Original Message -----
> >>> From: "Gleb Peregud" <gleber.p@REDACTED>
> >>> To: "Vlad Dumitrescu" <vladdu55@REDACTED>
> >>> Cc: "Erlang" <erlang-questions@REDACTED>
> >>> Sent: Monday, 19 November, 2012 2:18:11 PM
> >>> Subject: Re: [erlang-questions] Futures/promises and ability to
> >>> "call" abstract modules
> >>>
> >>> With the these changes [1] the following code works as intended:
> >>>
> >>> 7> F = future:new(fun() -> timer:sleep(5000), 42 end).
> >>> {future,<0.44.0>,#Ref<0.0.0.71>,undefined}
> >>> 8> F().
> >>> 42
> >>>
> >>> Aside from a problem where it improperly reports arity in case of
> >>> undefined function:
> >>>
> >>> 9> F(1,2,3).
> >>> ** exception error: undefined function future:call/3
> >>>
> >>> 1: https://github.com/gleber/otp/compare/call-abstract-module
> >>>
> >>> On Mon, Nov 19, 2012 at 11:48 AM, Gleb Peregud <gleber.p@REDACTED>
> >>> wrote:
> >>>> Sverker Eriksson wrote the following in [1]:
> >>>>
> >>>>> But even if the resource terms look alike, they are unique and
> >>>>> there is no bug leaking NIF resources (that I know of). A resource
> >>>>> is released (and destructor called) when the last reference is
> >>>>> garbage collected.
> >>>>> The shell can fool you however, as it keeps a command history that
> >>>>> can retain terms even though you think the variables are
> >>>>> forgotten.
> >>>>> Test NIF
> >>>>> resource cleanup by running a test module and call
> >>>>> erlang:garbage_collect to force destructors to be called.
> >>>>
> >>>> This seems to mean that they are "shared" and garbage collected
> >>>> just once.
> >>>>
> >>>> 1:
> >>>> http://erlang.org/pipermail/erlang-questions/2011-January/055524.ht
> >>>> ml
> >>>>
> >>>> On Mon, Nov 19, 2012 at 11:46 AM, Vlad Dumitrescu
> >>>> <vladdu55@REDACTED> wrote:
> >>>>> I have no idea, that's why I asked :-) /Vlad
> >>>>>
> >>>>>
> >>>>> On Mon, Nov 19, 2012 at 11:44 AM, Gleb Peregud
> >>>>> <gleber.p@REDACTED> wrote:
> >>>>>>
> >>>>>> I assumed that NIF-generated resources are shared between
> >>>>>> processes (the same way as large binaries are), and I haven't
> >>>>>> done any tests on this. Are you sure it is garbate collected
> >>>>>> multiple times (once per referencing process)?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Nov 19, 2012 at 11:41 AM, Vlad Dumitrescu
> >>>>>> <vladdu55@REDACTED>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Hi Gleb,
> >>>>>>>
> >>>>>>> just a quick observation about garbage collecting futures: would
> >>>>>>> the NIF-generated resource keep track of usage across processes?
> >>>>>>> I fI send a future as a message, it may be referenced by
> >>>>>>> multiple processes which have their own heap and garbage
> >>>>>>> collection...
> >>>>>>>
> >>>>>>> regards,
> >>>>>>> Vlad
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Nov 19, 2012 at 11:32 AM, Gleb Peregud
> >>>>>>> <gleber.p@REDACTED>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hello
> >>>>>>>>
> >>>>>>>> Last evening I was trying to implement futures/promise
> >>>>>>>> mechanism in Erlang (mostly for fun, since I am still unsure if
> >>>>>>>> it is useful). I got inspired with the presentation [1], which
> >>>>>>>> mentioned using futures as a foundation of building services,
> >>>>>>>> where things like timeouts, tracing, authentication, etc. is
> >>>>>>>> built by composing futures (see slide 41).
> >>>>>>>>
> >>>>>>>> Do you think that such composition of futures could be useful
> >>>>>>>> as a tool to improve code reuse of communication patterns in
> >>>>>>>> Erlang (as described in the presentation)?
> >>>>>>>>
> >>>>>>>> I've implemented futures using processes and message passing
> >>>>>>>> and stumbled upon two issues:
> >>>>>>>> 1) garbage collection of futures
> >>>>>>>> 2) slightly too much code when using them
> >>>>>>>>
> >>>>>>>> Example of the first problem is here:
> >>>>>>>>
> >>>>>>>> 1> F = future:new(fun() -> timer:sleep(10000), 10 end).
> >>>>>>>> {future,<0.36.0>,#Ref<0.0.0.1736>,undefined}
> >>>>>>>> 2> F:get(). %% it hangs for 10 seconds
> >>>>>>>> 10
> >>>>>>>>
> >>>>>>>> Since future F is represented as a process <0.36.0> it will
> >>>>>>>> stay running forever till it's timed out (which is not a good
> >>>>>>>> solution, since someone may still have a reference to this
> >>>>>>>> future) or F:done() manually called.
> >>>>>>>>
> >>>>>>>> My idea is to insert into 'future' tuple a NIF-generated
> >>>>>>>> resource, which will have a destructor attached (called upon
> >>>>>>>> garbage collection of the
> >>>>>>>> resource) which will call F:done(). Will it work?
> >>>>>>>>
> >>>>>>>> The second issue is illustrated here:
> >>>>>>>>
> >>>>>>>> 7> F = future:new().
> >>>>>>>> {future,<0.47.0>,#Ref<0.0.0.27235>,undefined}
> >>>>>>>> 8> spawn(fun() -> timer:sleep(10000), F:set(42) end).
> >>>>>>>> <0.49.0>
> >>>>>>>> 9> F:get().
> >>>>>>>> 42
> >>>>>>>>
> >>>>>>>> In ideal world it should be enough to just write "F" (without
> >>>>>>>> :get()) to
> >>>>>>>> fetch future's value, but it seems too far fetched for Erlang.
> >>>>>>>> Slightly
> >>>>>>>> better solution would be to allow calling future with "F()".
> >>>>>>>>
> >>>>>>>> This can be done by extending concept of "abstract modules"
> >>>>>>>> with
> >>>>>>>> "default call". Currently abstract modules allow the following:
> >>>>>>>>
> >>>>>>>> {future, Pid, Ref, undefined}:get() which is translated to
> >>>>>>>> future:get({future, Pid, Ref, undefined})
> >>>>>>>>
> >>>>>>>> With a simple change in beam_emu.c in call_fun function (which
> >>>>>>>> would replace obsolete fun tuples) we can allow for the
> >>>>>>>> following:
> >>>>>>>>
> >>>>>>>> {future, Pid, Ref, undefined}() which COULD be translated to
> >>>>>>>> future:call({future, Pid, Ref, undefined})
> >>>>>>>>
> >>>>>>>> hence allowing to use just "F()" to read a value of the future.
> >>>>>>>> This
> >>>>>>>> will also extend "metaprogramming" capabilities of Erlang for
> >>>>>>>> some other quirky use, which may or may not be a Good
> >>>>>>>> Thing(tm).
> >>>>>>>>
> >>>>>>>> Thoughts?
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Gleb Peregud
> >>>>>>>>
> >>>>>>>> 1: http://monkey.org/~marius/talks/twittersystems/
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> erlang-questions mailing list
> >>>>>>>> erlang-questions@REDACTED
> >>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>> _______________________________________________
> >>> erlang-questions mailing list
> >>> erlang-questions@REDACTED
> >>> http://erlang.org/mailman/listinfo/erlang-questions
> >>>
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions