[erlang-questions] Futures/promises and ability to "call" abstract modules

[Gleb Peregud]

Garret, my initial motivation of futures can be actually found at
slide 41-44. It is the ability to compose/stack some generic behaviors
like retries, sharding, timeouts upon some basic operations without
knowing details of those operations.

I've implemented "retry", "timeout" and "safe" wrappers in erlfu to
see if it is feasible concept. Let's assume you have a DB server which
has a very unstable connection, but you want to make sure to get some
results from that server.

Here's example basic code for fetching results from that server:

fetch(Key, Host, Port) ->
   {ok, S} = get_tcp:connect(Host, Port, [line]),
   gen_tcp:send(S, ["GET ", Key, "\n\n"]),
   {ok, Result} = gen_tcp:recv(S, 0),
   list_to_integer(Result).

And you have some sort of server who manages connection to that server:

handle_call({fetch, Key}, #state{host = Host, port = Port} = State) ->
   Result = fetch(Key, Host, Port),
   {reply, Result, State}.

but since remote server is very unstable there's high probabilty that
gen_server:calls will timeout and the gen_server will block for a long
time. So you rewrite it like this (I know I could use noreply, spawn,
gen_server:reply combo for this, but bear with me):

handle_call({fetch, Key}, #state{host = Host, port = Port} = State) ->
   F = future:new(fun() -> fetch(Key, Host, Port) end),
   {reply, F, State}.

and make client run F:get() when it needs actual data.

This makes gen_server independent of the unstable remote DB server,
which makes things a bit better. But it's not enough, since connection
to DB server is very unstable, hence you want to add timeouts and
retries, and you can do it easily by rewriting that like this:

handle_call({fetch, Key}, #state{host = Host, port = Port} = State) ->
   F = future:new(fun() -> fetch(Key, Host, Port) end),
   F2 = future:timeout(F, 5000),
   F3 = future:retry(F2, 5),
   {reply, F, State}.

It's a two-lines change but it "automagically" makes these requests do
5 retries with 5s timeout each.

This example is a bit artificial, but gives some idea about possible
use of those "futures". Other uses which seems possible are generic
sharding behavior, stats gathering, tracing and logging. Which
essentially gives ability to easily reuse some communication-related
patters with very little code repetition.

P.S. http://github.com/gleber/erlfu now uses gcproc and resource
projects to make futures garbage collected (with a limitation that it
works only inside one node).

On Mon, Nov 19, 2012 at 10:40 PM, Garrett Smith <g@REDACTED> wrote:
> On Mon, Nov 19, 2012 at 4:32 AM, Gleb Peregud <gleber.p@REDACTED> wrote:
>> Hello
>>
>> Last evening I was trying to implement futures/promise mechanism in Erlang
>> (mostly for fun, since I am still unsure if it is useful). I got inspired
>> with the presentation [1], which mentioned using futures as a foundation of
>> building services, where things like timeouts, tracing, authentication, etc.
>> is built by composing futures (see slide 41).
>>
>> Do you think that such composition of futures could be useful as a tool to
>> improve code reuse of communication patterns in Erlang (as described in the
>> presentation)?
>
> From the presentation, the only motivation for "futures" that I can see is this:
>
> http://monkey.org/~marius/talks/twittersystems/#26
>
> Here's what's on that slide, entitled "Futures":
>
> * A placeholder for for a result that is, usually, being computed concurrently
> -- long computations
> -- network call
> -- reading from disk
> * Computations can fail:
> -- connection failure
> -- timeout
> -- div by zero
> * Futures are how we represent concurrent execution.
>
> I don't see anywhere a problem here, unless you're unhappy with
> threads and want to use some "async" library that requires the use of
> callbacks, or this futures thing.
>
> Of course that's why you'd use Erlang.
>
> As it turns out, all of this is handled rather elegantly by Erlang's
> concurrency model, which uses processes and message passing.
>
> It might be an interesting exercise to figure out how GC works in
> under a particular use case, but from the list above, it's unclear
> what problems you're trying to solve in Erlang.
>
> Garrett