[erlang-questions] Futures/promises and ability to "call" abstract modules

Mon Nov 19 22:06:45 CET 2012

In this particular case of futures implementation, using NIF-based
resources, it would be enough to send a {done, Ref} to a process and
it'll handle it and cease any further looping (hence terminating
itself), which seems elegant enough. Killing is also an option, but
brutality is not necessary in my case :)

Of course distributed case, as mentioned, is a whole new story.

On Mon, Nov 19, 2012 at 9:58 PM, David Mercer <dmercer@REDACTED> wrote:
> At a high level, how would you GC the processes?  Just kill processes stuck at a receive (with no after) for which no other process on any connected node have a reference to the process?  Or is there a more elegant/better way to do it?
>
> Cheers,
>
> DBM
>
>> -----Original Message-----
>> From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-
>> bounces@REDACTED] On Behalf Of Gleb Peregud
>> Sent: Monday, November 19, 2012 13:06
>> To: Patrik Nyblom
>> Cc: Erlang
>> Subject: Re: [erlang-questions] Futures/promises and ability to "call"
>> abstract modules
>>
>> Thanks for info :) Distributed version, if implemented nicely, would
>> probably need changes in ERTS (e.g. ability to send resources between
>> nodes and call some callbacks upon receive); otherwise users of library
>> would need to manually call something like F:register() upon every
>> receive...
>>
>> Actually garbage collection of processes may be useful as a standalone
>> library - I had few situations when I wanted it to exist (but always
>> managed to do the job without it).
>>
>> On Mon, Nov 19, 2012 at 2:38 PM, Patrik Nyblom <pan@REDACTED> wrote:
>> > On 11/19/2012 11:48 AM, Gleb Peregud wrote:
>> >>
>> >> Sverker Eriksson wrote the following in [1]:
>> >>
>> >>> But even if the resource terms look alike, they are unique and there
>> >>> is no bug leaking NIF resources (that I know of). A resource is
>> >>> released (and destructor called) when the last reference is garbage
>> collected.
>> >>> The shell can fool you however, as it keeps a command history that
>> >>> can retain terms even though you think the variables are forgotten.
>> >>> Test NIF resource cleanup by running a test module and call
>> >>> erlang:garbage_collect to force destructors to be called.
>> >>
>> >> This seems to mean that they are "shared" and garbage collected just
>> once.
>> >
>> > That is correct. The callback however, is not really able to send
>> > messages, you would have to relay information to a thread which in
>> > turn would have to send a message to the future to make it exit. It
>> > seems to be technically possible at least, given an SMP VM. You would
>> > more or less implement garbage collection of processes, which would be
>> > kind of cool, I think :)
>> >
>> > Of course distribution would generate a slightly bigger challenge...
>> >
>> > Cheers,
>> > /Patrik
>> >
>> >>
>> >> 1:
>> >> http://erlang.org/pipermail/erlang-questions/2011-January/055524.html
>> >>
>> >> On Mon, Nov 19, 2012 at 11:46 AM, Vlad Dumitrescu
>> >> <vladdu55@REDACTED>
>> >> wrote:
>> >>>
>> >>> I have no idea, that's why I asked :-) /Vlad
>> >>>
>> >>>
>> >>> On Mon, Nov 19, 2012 at 11:44 AM, Gleb Peregud <gleber.p@REDACTED>
>> >>> wrote:
>> >>>>
>> >>>> I assumed that NIF-generated resources are shared between processes
>> (the
>> >>>> same way as large binaries are), and I haven't done any tests on
>> this.
>> >>>> Are
>> >>>> you sure it is garbate collected multiple times (once per referencing
>> >>>> process)?
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Mon, Nov 19, 2012 at 11:41 AM, Vlad Dumitrescu
>> <vladdu55@REDACTED>
>> >>>> wrote:
>> >>>>>
>> >>>>> Hi Gleb,
>> >>>>>
>> >>>>> just a quick observation about garbage collecting futures: would the
>> >>>>> NIF-generated resource keep track of usage across processes? I fI
>> send
>> >>>>> a
>> >>>>> future as a message, it may be referenced by multiple processes
>> which
>> >>>>> have
>> >>>>> their own heap and garbage collection...
>> >>>>>
>> >>>>> regards,
>> >>>>> Vlad
>> >>>>>
>> >>>>>
>> >>>>> On Mon, Nov 19, 2012 at 11:32 AM, Gleb Peregud <gleber.p@REDACTED>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> Hello
>> >>>>>>
>> >>>>>> Last evening I was trying to implement futures/promise mechanism in
>> >>>>>> Erlang (mostly for fun, since I am still unsure if it is useful). I
>> >>>>>> got
>> >>>>>> inspired with the presentation [1], which mentioned using futures
>> as a
>> >>>>>> foundation of building services, where things like timeouts,
>> tracing,
>> >>>>>> authentication, etc. is built by composing futures (see slide 41).
>> >>>>>>
>> >>>>>> Do you think that such composition of futures could be useful as a
>> >>>>>> tool
>> >>>>>> to improve code reuse of communication patterns in Erlang (as
>> >>>>>> described in
>> >>>>>> the presentation)?
>> >>>>>>
>> >>>>>> I've implemented futures using processes and message passing and
>> >>>>>> stumbled upon two issues:
>> >>>>>> 1) garbage collection of futures
>> >>>>>> 2) slightly too much code when using them
>> >>>>>>
>> >>>>>> Example of the first problem is here:
>> >>>>>>
>> >>>>>> 1> F = future:new(fun() -> timer:sleep(10000), 10 end).
>> >>>>>> {future,<0.36.0>,#Ref<0.0.0.1736>,undefined}
>> >>>>>> 2> F:get(). %% it hangs for 10 seconds
>> >>>>>> 10
>> >>>>>>
>> >>>>>> Since future F is represented as a process <0.36.0> it will stay
>> >>>>>> running
>> >>>>>> forever till it's timed out (which is not a good solution, since
>> >>>>>> someone may
>> >>>>>> still have a reference to this future) or F:done() manually called.
>> >>>>>>
>> >>>>>> My idea is to insert into 'future' tuple a NIF-generated resource,
>> >>>>>> which
>> >>>>>> will have a destructor attached (called upon garbage collection of
>> the
>> >>>>>> resource) which will call F:done(). Will it work?
>> >>>>>>
>> >>>>>> The second issue is illustrated here:
>> >>>>>>
>> >>>>>> 7> F = future:new().
>> >>>>>> {future,<0.47.0>,#Ref<0.0.0.27235>,undefined}
>> >>>>>> 8> spawn(fun() -> timer:sleep(10000), F:set(42) end).
>> >>>>>> <0.49.0>
>> >>>>>> 9> F:get().
>> >>>>>> 42
>> >>>>>>
>> >>>>>> In ideal world it should be enough to just write "F" (without
>> :get())
>> >>>>>> to
>> >>>>>> fetch future's value, but it seems too far fetched for Erlang.
>> >>>>>> Slightly
>> >>>>>> better solution would be to allow calling future with "F()".
>> >>>>>>
>> >>>>>> This can be done by extending concept of "abstract modules" with
>> >>>>>> "default call". Currently abstract modules allow the following:
>> >>>>>>
>> >>>>>> {future, Pid, Ref, undefined}:get() which is translated to
>> >>>>>> future:get({future, Pid, Ref, undefined})
>> >>>>>>
>> >>>>>> With a simple change in beam_emu.c in call_fun function (which
>> would
>> >>>>>> replace obsolete fun tuples) we can allow for the following:
>> >>>>>>
>> >>>>>> {future, Pid, Ref, undefined}() which COULD be translated to
>> >>>>>> future:call({future, Pid, Ref, undefined})
>> >>>>>>
>> >>>>>> hence allowing to use just "F()" to read a value of the future.
>> This
>> >>>>>> will also extend "metaprogramming" capabilities of Erlang for some
>> >>>>>> other
>> >>>>>> quirky use, which may or may not be a Good Thing(tm).
>> >>>>>>
>> >>>>>> Thoughts?
>> >>>>>>
>> >>>>>> Cheers,
>> >>>>>> Gleb Peregud
>> >>>>>>
>> >>>>>> 1: http://monkey.org/~marius/talks/twittersystems/
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> erlang-questions mailing list
>> >>>>>> erlang-questions@REDACTED
>> >>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>> >>>>>>
>> >> _______________________________________________
>> >> erlang-questions mailing list
>> >> erlang-questions@REDACTED
>> >> http://erlang.org/mailman/listinfo/erlang-questions
>> >
>> >
>> > _______________________________________________
>> > erlang-questions mailing list
>> > erlang-questions@REDACTED
>> > http://erlang.org/mailman/listinfo/erlang-questions
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>