[erlang-questions] Futures/promises and ability to "call" abstract modules

Tony Rogvall tony@REDACTED
Tue Nov 20 17:16:18 CET 2012 

On 20 nov 2012, at 16:20, Gleb Peregud <gleber.p@REDACTED> wrote:

> This is yet to be implemented. The idea is to rely on already existing
> implementation of NIF resources - this implies that a user of the
> library would have to always store Pid of garbage collectable process
> (let's call it gcproc) along with that resource in his heap, for
> example as {'gcproc', pid(), resource()} and use special API to do
> sending, killing, linking, monitoring, etc.
> 
> Since the resource associated with the process is always stored in
> heap of the process, the resource will not get garbage collected until
> no heap references it. This way we tie together process and resource
> lifetime. And since NIF resources can have a destructor in C which is
> called upon resource is garbage collected, we can make use of it to
> inform gcproc to terminate.
> 
> As Patrik pointed out it is not as trivial as it sounds, but still possible.
> 
Maybe git://github.com/tonyrog/resource.git could come in handy.

/Tony

> On Tue, Nov 20, 2012 at 3:34 PM, David Mercer <dmercer@REDACTED> wrote:
>> How does that spawned process get garbage collected?
>> 
>> Cheers,
>> 
>> DBM
>> 
>> 
>>> -----Original Message-----
>>> From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-
>>> bounces@REDACTED] On Behalf Of Dmitry Belyaev
>>> Sent: Monday, November 19, 2012 21:48
>>> To: Gleb Peregud
>>> Cc: Erlang
>>> Subject: Re: [erlang-questions] Futures/promises and ability to "call"
>>> abstract modules
>>> 
>>> Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:2:2] [async-threads:0]
>>> [hipe] [kernel-poll:false]
>>> 
>>> Eshell V5.9.1  (abort with ^G)
>>> 1> F = (fun() -> FM = fun(M, V) -> receive {value, Value} -> M(M, Value);
>>> {get, P} -> P ! V, M(M, V) end end, P = spawn(fun() -> FM(FM, undefined)
>>> end), fun() -> P ! {get, self()}, receive V -> V end end end)().
>>> #Fun<erl_eval.20.82930912>
>>> 2> F().
>>> undefined
>>> 3> element(2,lists:keyfind('P', 1,
>>> lists:flatten(element(2,erlang:fun_info(F, env))))) ! {value, 10}.
>>> {value,10}
>>> 4> F().
>>> 10
>>> 5>
>>> 
>>> --
>>> Dmitry Belyaev
>>> 
>>> On 19.11.2012, at 18:05, Gleb Peregud wrote:
>>> 
>>>> Then I wouldn't be able to set future's value after it being defined.
>>>> Like in this example:
>>>> 
>>>> 7> F = future:new().
>>>> {future,<0.47.0>,#Ref<0.0.0.27235>,undefined}
>>>> 8> spawn(fun() -> timer:sleep(10000), F:set(42) end).
>>>> <0.49.0>
>>>> 9> F:get().
>>>> 42
>>>> 
>>>> Without this ability such futures are useless with request-reply
>>> pattern:
>>>> 1) Client sends request
>>>> 2) Server creates a future and sends it back to client
>>>> 3) Client does it's work with the value of the future (without
>>>> bothering the fact that value may become available in uncertain
>>>> future)
>>>> 4) Server finishes computations and sets future's value
>>>> 
>>>> 
>>>> On Mon, Nov 19, 2012 at 3:01 PM, Robert Virding
>>>> <robert.virding@REDACTED> wrote:
>>>>> Wouldn't it be easier if future:new just returned a fun? Then you could
>>> do F() without any changes?
>>>>> 
>>>>> Robert
>>>>> 
>>>>> ----- Original Message -----
>>>>>> From: "Gleb Peregud" <gleber.p@REDACTED>
>>>>>> To: "Vlad Dumitrescu" <vladdu55@REDACTED>
>>>>>> Cc: "Erlang" <erlang-questions@REDACTED>
>>>>>> Sent: Monday, 19 November, 2012 2:18:11 PM
>>>>>> Subject: Re: [erlang-questions] Futures/promises and ability to
>>>>>> "call" abstract modules
>>>>>> 
>>>>>> With the these changes [1] the following code works as intended:
>>>>>> 
>>>>>> 7> F = future:new(fun() -> timer:sleep(5000), 42 end).
>>>>>> {future,<0.44.0>,#Ref<0.0.0.71>,undefined}
>>>>>> 8> F().
>>>>>> 42
>>>>>> 
>>>>>> Aside from a problem where it improperly reports arity in case of
>>>>>> undefined function:
>>>>>> 
>>>>>> 9> F(1,2,3).
>>>>>> ** exception error: undefined function future:call/3
>>>>>> 
>>>>>> 1: https://github.com/gleber/otp/compare/call-abstract-module
>>>>>> 
>>>>>> On Mon, Nov 19, 2012 at 11:48 AM, Gleb Peregud <gleber.p@REDACTED>
>>>>>> wrote:
>>>>>>> Sverker Eriksson wrote the following in [1]:
>>>>>>> 
>>>>>>>> But even if the resource terms look alike, they are unique and
>>>>>>>> there is no bug leaking NIF resources (that I know of). A resource
>>>>>>>> is released (and destructor called) when the last reference is
>>>>>>>> garbage collected.
>>>>>>>> The shell can fool you however, as it keeps a command history that
>>>>>>>> can retain terms even though you think the variables are
>>>>>>>> forgotten.
>>>>>>>> Test NIF
>>>>>>>> resource cleanup by running a test module and call
>>>>>>>> erlang:garbage_collect to force destructors to be called.
>>>>>>> 
>>>>>>> This seems to mean that they are "shared" and garbage collected
>>>>>>> just once.
>>>>>>> 
>>>>>>> 1:
>>>>>>> http://erlang.org/pipermail/erlang-questions/2011-January/055524.ht
>>>>>>> ml
>>>>>>> 
>>>>>>> On Mon, Nov 19, 2012 at 11:46 AM, Vlad Dumitrescu
>>>>>>> <vladdu55@REDACTED> wrote:
>>>>>>>> I have no idea, that's why I asked :-) /Vlad
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, Nov 19, 2012 at 11:44 AM, Gleb Peregud
>>>>>>>> <gleber.p@REDACTED> wrote:
>>>>>>>>> 
>>>>>>>>> I assumed that NIF-generated resources are shared between
>>>>>>>>> processes (the same way as large binaries are), and I haven't
>>>>>>>>> done any tests on this. Are you sure it is garbate collected
>>>>>>>>> multiple times (once per referencing process)?
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Mon, Nov 19, 2012 at 11:41 AM, Vlad Dumitrescu
>>>>>>>>> <vladdu55@REDACTED>
>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hi Gleb,
>>>>>>>>>> 
>>>>>>>>>> just a quick observation about garbage collecting futures: would
>>>>>>>>>> the NIF-generated resource keep track of usage across processes?
>>>>>>>>>> I fI send a future as a message, it may be referenced by
>>>>>>>>>> multiple processes which have their own heap and garbage
>>>>>>>>>> collection...
>>>>>>>>>> 
>>>>>>>>>> regards,
>>>>>>>>>> Vlad
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Mon, Nov 19, 2012 at 11:32 AM, Gleb Peregud
>>>>>>>>>> <gleber.p@REDACTED>
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hello
>>>>>>>>>>> 
>>>>>>>>>>> Last evening I was trying to implement futures/promise
>>>>>>>>>>> mechanism in Erlang (mostly for fun, since I am still unsure if
>>>>>>>>>>> it is useful). I got inspired with the presentation [1], which
>>>>>>>>>>> mentioned using futures as a foundation of building services,
>>>>>>>>>>> where things like timeouts, tracing, authentication, etc. is
>>>>>>>>>>> built by composing futures (see slide 41).
>>>>>>>>>>> 
>>>>>>>>>>> Do you think that such composition of futures could be useful
>>>>>>>>>>> as a tool to improve code reuse of communication patterns in
>>>>>>>>>>> Erlang (as described in the presentation)?
>>>>>>>>>>> 
>>>>>>>>>>> I've implemented futures using processes and message passing
>>>>>>>>>>> and stumbled upon two issues:
>>>>>>>>>>> 1) garbage collection of futures
>>>>>>>>>>> 2) slightly too much code when using them
>>>>>>>>>>> 
>>>>>>>>>>> Example of the first problem is here:
>>>>>>>>>>> 
>>>>>>>>>>> 1> F = future:new(fun() -> timer:sleep(10000), 10 end).
>>>>>>>>>>> {future,<0.36.0>,#Ref<0.0.0.1736>,undefined}
>>>>>>>>>>> 2> F:get(). %% it hangs for 10 seconds
>>>>>>>>>>> 10
>>>>>>>>>>> 
>>>>>>>>>>> Since future F is represented as a process <0.36.0> it will
>>>>>>>>>>> stay running forever till it's timed out (which is not a good
>>>>>>>>>>> solution, since someone may still have a reference to this
>>>>>>>>>>> future) or F:done() manually called.
>>>>>>>>>>> 
>>>>>>>>>>> My idea is to insert into 'future' tuple a NIF-generated
>>>>>>>>>>> resource, which will have a destructor attached (called upon
>>>>>>>>>>> garbage collection of the
>>>>>>>>>>> resource) which will call F:done(). Will it work?
>>>>>>>>>>> 
>>>>>>>>>>> The second issue is illustrated here:
>>>>>>>>>>> 
>>>>>>>>>>> 7> F = future:new().
>>>>>>>>>>> {future,<0.47.0>,#Ref<0.0.0.27235>,undefined}
>>>>>>>>>>> 8> spawn(fun() -> timer:sleep(10000), F:set(42) end).
>>>>>>>>>>> <0.49.0>
>>>>>>>>>>> 9> F:get().
>>>>>>>>>>> 42
>>>>>>>>>>> 
>>>>>>>>>>> In ideal world it should be enough to just write "F" (without
>>>>>>>>>>> :get()) to
>>>>>>>>>>> fetch future's value, but it seems too far fetched for Erlang.
>>>>>>>>>>> Slightly
>>>>>>>>>>> better solution would be to allow calling future with "F()".
>>>>>>>>>>> 
>>>>>>>>>>> This can be done by extending concept of "abstract modules"
>>>>>>>>>>> with
>>>>>>>>>>> "default call". Currently abstract modules allow the following:
>>>>>>>>>>> 
>>>>>>>>>>> {future, Pid, Ref, undefined}:get() which is translated to
>>>>>>>>>>> future:get({future, Pid, Ref, undefined})
>>>>>>>>>>> 
>>>>>>>>>>> With a simple change in beam_emu.c in call_fun function (which
>>>>>>>>>>> would replace obsolete fun tuples) we can allow for the
>>>>>>>>>>> following:
>>>>>>>>>>> 
>>>>>>>>>>> {future, Pid, Ref, undefined}() which COULD be translated to
>>>>>>>>>>> future:call({future, Pid, Ref, undefined})
>>>>>>>>>>> 
>>>>>>>>>>> hence allowing to use just "F()" to read a value of the future.
>>>>>>>>>>> This
>>>>>>>>>>> will also extend "metaprogramming" capabilities of Erlang for
>>>>>>>>>>> some other quirky use, which may or may not be a Good
>>>>>>>>>>> Thing(tm).
>>>>>>>>>>> 
>>>>>>>>>>> Thoughts?
>>>>>>>>>>> 
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Gleb Peregud
>>>>>>>>>>> 
>>>>>>>>>>> 1: http://monkey.org/~marius/talks/twittersystems/
>>>>>>>>>>> 
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> erlang-questions mailing list
>>>>>>>>>>> erlang-questions@REDACTED
>>>>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>> _______________________________________________
>>>>>> erlang-questions mailing list
>>>>>> erlang-questions@REDACTED
>>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>> 
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>> 
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

"Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix"



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121120/61c045db/attachment.htm>

More information about the erlang-questions mailing list