[erlang-questions] Futures/promises and ability to "call" abstract modules

[Gleb Peregud]

Now that example actually work. Previously retries were not working
due to future cloning worked only one level deep (it was a shallow).
I've improved it to do deep copying and now, along with automatic
garbage collection, it can be used in practice.

It was a great exercise :) Let me know if anyone *actually* use it ;)

On Tue, Nov 20, 2012 at 7:59 PM, Gleb Peregud <gleber.p@REDACTED> wrote:
> Garret, my initial motivation of futures can be actually found at
> slide 41-44. It is the ability to compose/stack some generic behaviors
> like retries, sharding, timeouts upon some basic operations without
> knowing details of those operations.
>
> I've implemented "retry", "timeout" and "safe" wrappers in erlfu to
> see if it is feasible concept. Let's assume you have a DB server which
> has a very unstable connection, but you want to make sure to get some
> results from that server.
>
> Here's example basic code for fetching results from that server:
>
> fetch(Key, Host, Port) ->
>    {ok, S} = get_tcp:connect(Host, Port, [line]),
>    gen_tcp:send(S, ["GET ", Key, "\n\n"]),
>    {ok, Result} = gen_tcp:recv(S, 0),
>    list_to_integer(Result).
>
> And you have some sort of server who manages connection to that server:
>
> handle_call({fetch, Key}, #state{host = Host, port = Port} = State) ->
>    Result = fetch(Key, Host, Port),
>    {reply, Result, State}.
>
> but since remote server is very unstable there's high probabilty that
> gen_server:calls will timeout and the gen_server will block for a long
> time. So you rewrite it like this (I know I could use noreply, spawn,
> gen_server:reply combo for this, but bear with me):
>
> handle_call({fetch, Key}, #state{host = Host, port = Port} = State) ->
>    F = future:new(fun() -> fetch(Key, Host, Port) end),
>    {reply, F, State}.
>
> and make client run F:get() when it needs actual data.
>
> This makes gen_server independent of the unstable remote DB server,
> which makes things a bit better. But it's not enough, since connection
> to DB server is very unstable, hence you want to add timeouts and
> retries, and you can do it easily by rewriting that like this:
>
> handle_call({fetch, Key}, #state{host = Host, port = Port} = State) ->
>    F = future:new(fun() -> fetch(Key, Host, Port) end),
>    F2 = future:timeout(F, 5000),
>    F3 = future:retry(F2, 5),
>    {reply, F, State}.
>
> It's a two-lines change but it "automagically" makes these requests do
> 5 retries with 5s timeout each.
>
> This example is a bit artificial, but gives some idea about possible
> use of those "futures". Other uses which seems possible are generic
> sharding behavior, stats gathering, tracing and logging. Which
> essentially gives ability to easily reuse some communication-related
> patters with very little code repetition.
>
> P.S. http://github.com/gleber/erlfu now uses gcproc and resource
> projects to make futures garbage collected (with a limitation that it
> works only inside one node).
>
> On Mon, Nov 19, 2012 at 10:40 PM, Garrett Smith <g@REDACTED> wrote:
>> On Mon, Nov 19, 2012 at 4:32 AM, Gleb Peregud <gleber.p@REDACTED> wrote:
>>> Hello
>>>
>>> Last evening I was trying to implement futures/promise mechanism in Erlang
>>> (mostly for fun, since I am still unsure if it is useful). I got inspired
>>> with the presentation [1], which mentioned using futures as a foundation of
>>> building services, where things like timeouts, tracing, authentication, etc.
>>> is built by composing futures (see slide 41).
>>>
>>> Do you think that such composition of futures could be useful as a tool to
>>> improve code reuse of communication patterns in Erlang (as described in the
>>> presentation)?
>>
>> From the presentation, the only motivation for "futures" that I can see is this:
>>
>> http://monkey.org/~marius/talks/twittersystems/#26
>>
>> Here's what's on that slide, entitled "Futures":
>>
>> * A placeholder for for a result that is, usually, being computed concurrently
>> -- long computations
>> -- network call
>> -- reading from disk
>> * Computations can fail:
>> -- connection failure
>> -- timeout
>> -- div by zero
>> * Futures are how we represent concurrent execution.
>>
>> I don't see anywhere a problem here, unless you're unhappy with
>> threads and want to use some "async" library that requires the use of
>> callbacks, or this futures thing.
>>
>> Of course that's why you'd use Erlang.
>>
>> As it turns out, all of this is handled rather elegantly by Erlang's
>> concurrency model, which uses processes and message passing.
>>
>> It might be an interesting exercise to figure out how GC works in
>> under a particular use case, but from the list above, it's unclear
>> what problems you're trying to solve in Erlang.
>>
>> Garrett