[erlang-questions] Futures/promises and ability to "call" abstract modules

Tue Nov 20 16:39:13 CET 2012

Dmitry, this is an interesting idea, but feels a bit hacky :)
Unfortunately I don't think that (with this approach)
resource:notify_when_destroyed/2 will help to implement automatic
garbage collection.

In contrast with abstract modules, where it should be possible to
implement. Of course without new implicit call user would still have
to use "F:get()" to get future value.

On Tue, Nov 20, 2012 at 4:32 PM, Dmitry Belyaev <be.dmitry@REDACTED> wrote:
> This is not real implementation. I just showed that it is possible to use fun environment outside of it. That way no parameterized modules and new implicit call to them are required.
> Of course garbage collection is the problem. Right now I don't have any good enough proposal to solve it.
>
> --
> Dmitry Belyaev
>
> On 20.11.2012, at 18:34, David Mercer wrote:
>
>> How does that spawned process get garbage collected?
>>
>> Cheers,
>>
>> DBM
>>
>>
>>> -----Original Message-----
>>> From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-
>>> bounces@REDACTED] On Behalf Of Dmitry Belyaev
>>> Sent: Monday, November 19, 2012 21:48
>>> To: Gleb Peregud
>>> Cc: Erlang
>>> Subject: Re: [erlang-questions] Futures/promises and ability to "call"
>>> abstract modules
>>>
>>> Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:2:2] [async-threads:0]
>>> [hipe] [kernel-poll:false]
>>>
>>> Eshell V5.9.1  (abort with ^G)
>>> 1> F = (fun() -> FM = fun(M, V) -> receive {value, Value} -> M(M, Value);
>>> {get, P} -> P ! V, M(M, V) end end, P = spawn(fun() -> FM(FM, undefined)
>>> end), fun() -> P ! {get, self()}, receive V -> V end end end)().
>>> #Fun<erl_eval.20.82930912>
>>> 2> F().
>>> undefined
>>> 3> element(2,lists:keyfind('P', 1,
>>> lists:flatten(element(2,erlang:fun_info(F, env))))) ! {value, 10}.
>>> {value,10}
>>> 4> F().
>>> 10
>>> 5>
>>>
>>> --
>>> Dmitry Belyaev
>>>
>>> On 19.11.2012, at 18:05, Gleb Peregud wrote:
>>>
>>>> Then I wouldn't be able to set future's value after it being defined.
>>>> Like in this example:
>>>>
>>>> 7> F = future:new().
>>>> {future,<0.47.0>,#Ref<0.0.0.27235>,undefined}
>>>> 8> spawn(fun() -> timer:sleep(10000), F:set(42) end).
>>>> <0.49.0>
>>>> 9> F:get().
>>>> 42
>>>>
>>>> Without this ability such futures are useless with request-reply
>>> pattern:
>>>> 1) Client sends request
>>>> 2) Server creates a future and sends it back to client
>>>> 3) Client does it's work with the value of the future (without
>>>> bothering the fact that value may become available in uncertain
>>>> future)
>>>> 4) Server finishes computations and sets future's value
>>>>
>>>>
>>>> On Mon, Nov 19, 2012 at 3:01 PM, Robert Virding
>>>> <robert.virding@REDACTED> wrote:
>>>>> Wouldn't it be easier if future:new just returned a fun? Then you could
>>> do F() without any changes?
>>>>>
>>>>> Robert
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Gleb Peregud" <gleber.p@REDACTED>
>>>>>> To: "Vlad Dumitrescu" <vladdu55@REDACTED>
>>>>>> Cc: "Erlang" <erlang-questions@REDACTED>
>>>>>> Sent: Monday, 19 November, 2012 2:18:11 PM
>>>>>> Subject: Re: [erlang-questions] Futures/promises and ability to
>>>>>> "call" abstract modules
>>>>>>
>>>>>> With the these changes [1] the following code works as intended:
>>>>>>
>>>>>> 7> F = future:new(fun() -> timer:sleep(5000), 42 end).
>>>>>> {future,<0.44.0>,#Ref<0.0.0.71>,undefined}
>>>>>> 8> F().
>>>>>> 42
>>>>>>
>>>>>> Aside from a problem where it improperly reports arity in case of
>>>>>> undefined function:
>>>>>>
>>>>>> 9> F(1,2,3).
>>>>>> ** exception error: undefined function future:call/3
>>>>>>
>>>>>> 1: https://github.com/gleber/otp/compare/call-abstract-module
>>>>>>
>>>>>> On Mon, Nov 19, 2012 at 11:48 AM, Gleb Peregud <gleber.p@REDACTED>
>>>>>> wrote:
>>>>>>> Sverker Eriksson wrote the following in [1]:
>>>>>>>
>>>>>>>> But even if the resource terms look alike, they are unique and
>>>>>>>> there is no bug leaking NIF resources (that I know of). A resource
>>>>>>>> is released (and destructor called) when the last reference is
>>>>>>>> garbage collected.
>>>>>>>> The shell can fool you however, as it keeps a command history that
>>>>>>>> can retain terms even though you think the variables are
>>>>>>>> forgotten.
>>>>>>>> Test NIF
>>>>>>>> resource cleanup by running a test module and call
>>>>>>>> erlang:garbage_collect to force destructors to be called.
>>>>>>>
>>>>>>> This seems to mean that they are "shared" and garbage collected
>>>>>>> just once.
>>>>>>>
>>>>>>> 1:
>>>>>>> http://erlang.org/pipermail/erlang-questions/2011-January/055524.ht
>>>>>>> ml
>>>>>>>
>>>>>>> On Mon, Nov 19, 2012 at 11:46 AM, Vlad Dumitrescu
>>>>>>> <vladdu55@REDACTED> wrote:
>>>>>>>> I have no idea, that's why I asked :-) /Vlad
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Nov 19, 2012 at 11:44 AM, Gleb Peregud
>>>>>>>> <gleber.p@REDACTED> wrote:
>>>>>>>>>
>>>>>>>>> I assumed that NIF-generated resources are shared between
>>>>>>>>> processes (the same way as large binaries are), and I haven't
>>>>>>>>> done any tests on this. Are you sure it is garbate collected
>>>>>>>>> multiple times (once per referencing process)?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Nov 19, 2012 at 11:41 AM, Vlad Dumitrescu
>>>>>>>>> <vladdu55@REDACTED>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Gleb,
>>>>>>>>>>
>>>>>>>>>> just a quick observation about garbage collecting futures: would
>>>>>>>>>> the NIF-generated resource keep track of usage across processes?
>>>>>>>>>> I fI send a future as a message, it may be referenced by
>>>>>>>>>> multiple processes which have their own heap and garbage
>>>>>>>>>> collection...
>>>>>>>>>>
>>>>>>>>>> regards,
>>>>>>>>>> Vlad
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Nov 19, 2012 at 11:32 AM, Gleb Peregud
>>>>>>>>>> <gleber.p@REDACTED>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hello
>>>>>>>>>>>
>>>>>>>>>>> Last evening I was trying to implement futures/promise
>>>>>>>>>>> mechanism in Erlang (mostly for fun, since I am still unsure if
>>>>>>>>>>> it is useful). I got inspired with the presentation [1], which
>>>>>>>>>>> mentioned using futures as a foundation of building services,
>>>>>>>>>>> where things like timeouts, tracing, authentication, etc. is
>>>>>>>>>>> built by composing futures (see slide 41).
>>>>>>>>>>>
>>>>>>>>>>> Do you think that such composition of futures could be useful
>>>>>>>>>>> as a tool to improve code reuse of communication patterns in
>>>>>>>>>>> Erlang (as described in the presentation)?
>>>>>>>>>>>
>>>>>>>>>>> I've implemented futures using processes and message passing
>>>>>>>>>>> and stumbled upon two issues:
>>>>>>>>>>> 1) garbage collection of futures
>>>>>>>>>>> 2) slightly too much code when using them
>>>>>>>>>>>
>>>>>>>>>>> Example of the first problem is here:
>>>>>>>>>>>
>>>>>>>>>>> 1> F = future:new(fun() -> timer:sleep(10000), 10 end).
>>>>>>>>>>> {future,<0.36.0>,#Ref<0.0.0.1736>,undefined}
>>>>>>>>>>> 2> F:get(). %% it hangs for 10 seconds
>>>>>>>>>>> 10
>>>>>>>>>>>
>>>>>>>>>>> Since future F is represented as a process <0.36.0> it will
>>>>>>>>>>> stay running forever till it's timed out (which is not a good
>>>>>>>>>>> solution, since someone may still have a reference to this
>>>>>>>>>>> future) or F:done() manually called.
>>>>>>>>>>>
>>>>>>>>>>> My idea is to insert into 'future' tuple a NIF-generated
>>>>>>>>>>> resource, which will have a destructor attached (called upon
>>>>>>>>>>> garbage collection of the
>>>>>>>>>>> resource) which will call F:done(). Will it work?
>>>>>>>>>>>
>>>>>>>>>>> The second issue is illustrated here:
>>>>>>>>>>>
>>>>>>>>>>> 7> F = future:new().
>>>>>>>>>>> {future,<0.47.0>,#Ref<0.0.0.27235>,undefined}
>>>>>>>>>>> 8> spawn(fun() -> timer:sleep(10000), F:set(42) end).
>>>>>>>>>>> <0.49.0>
>>>>>>>>>>> 9> F:get().
>>>>>>>>>>> 42
>>>>>>>>>>>
>>>>>>>>>>> In ideal world it should be enough to just write "F" (without
>>>>>>>>>>> :get()) to
>>>>>>>>>>> fetch future's value, but it seems too far fetched for Erlang.
>>>>>>>>>>> Slightly
>>>>>>>>>>> better solution would be to allow calling future with "F()".
>>>>>>>>>>>
>>>>>>>>>>> This can be done by extending concept of "abstract modules"
>>>>>>>>>>> with
>>>>>>>>>>> "default call". Currently abstract modules allow the following:
>>>>>>>>>>>
>>>>>>>>>>> {future, Pid, Ref, undefined}:get() which is translated to
>>>>>>>>>>> future:get({future, Pid, Ref, undefined})
>>>>>>>>>>>
>>>>>>>>>>> With a simple change in beam_emu.c in call_fun function (which
>>>>>>>>>>> would replace obsolete fun tuples) we can allow for the
>>>>>>>>>>> following:
>>>>>>>>>>>
>>>>>>>>>>> {future, Pid, Ref, undefined}() which COULD be translated to
>>>>>>>>>>> future:call({future, Pid, Ref, undefined})
>>>>>>>>>>>
>>>>>>>>>>> hence allowing to use just "F()" to read a value of the future.
>>>>>>>>>>> This
>>>>>>>>>>> will also extend "metaprogramming" capabilities of Erlang for
>>>>>>>>>>> some other quirky use, which may or may not be a Good
>>>>>>>>>>> Thing(tm).
>>>>>>>>>>>
>>>>>>>>>>> Thoughts?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Gleb Peregud
>>>>>>>>>>>
>>>>>>>>>>> 1: http://monkey.org/~marius/talks/twittersystems/
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> erlang-questions mailing list
>>>>>>>>>>> erlang-questions@REDACTED
>>>>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>> _______________________________________________
>>>>>> erlang-questions mailing list
>>>>>> erlang-questions@REDACTED
>>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>