[erlang-questions] Erlang and Akka, The Sequel

[Fred Hebert]

On 03/29, Youngkin, Rich wrote:
>Fred's response in [2] is interesting and informative, but it misses 
>the mark in that it's up to the developer to know/understand how and 
>when to implement the appropriate pattern. That's what capabilities 
>such as gen_server are intended to eliminate.

I'm not sure I understand the criticism there. [2] was about using 
RPC/futures, and the answer shows how to use RPC/futures.

Also there is *no replacement* to know when to use the appropriate 
pattern. I'm not sure of the intended wording here, if you meant 
"implement" as build the pattern from scratch, or "implement" as using 
the pattern. I went here with the latter interpretation, but do know 
that for futures, there is nothing else to do in Erlang than the example 
in my response to the initial post.

Let's use the Akka example (from 
http://doc.akka.io/docs/akka/snapshot/java/futures.html):

    Timeout timeout = new Timeout(Duration.create(5, "seconds"));
    Future<Object> future = Patterns.ask(actor, msg, timeout);
    String result = (String) Await.result(future, timeout.durtion())

Think of the following usage in Erlang:

   Timeout = timer:seconds(5000),
   Future = rpc:async_call(node(), Mod, call, [Actor, Msg, [Timeout]),
   {ok, Result} = rpc:nb_yield(Future, Timeout).

They are semantically equivalent. The only thing tricky about Erlang is 
to find it in the RPC module.

>1. The concept that an operation might fail (i.e., the "Try" type in 
>Akka).

It appears that `Try` is a type in Scala, not Akka. I went to 
http://helenaedelson.com/?p=939 and 
http://www.scala-lang.org/files/archive/nightly/docs/library/index.html#scala.util.Try 
to try to figure out what they are.

Those appear to be your standard 'either' type, and 'match' in the 
following example looks a bit like a 'maybe' monad, but I'll get on that 
later to avoid reordering the quotes of your email.

>2. Being able to easily compose operations that might fail in a way where
>the logic to detect and react to the failure can be expressed separately
>from the main logic path, aka the "happy path".
>

That does sound a lot like 'let it crash', but I'm guessing the point 
here is due to the cumbersome level of `{ok, Value}` vs. any other value 
to represent successes vs. errors (i.e. moving from an Either type to 
also passing them to a Maybe monad)

>Here's an example of the first (in Scala, taken from the Coursera course on
>Principles of Reactive Programming). It's simplistic, but I think it's
>applicable in more complicated use cases. Hopefully the Scala is pretty
>self-explanatory:
>
>1 val coins: Try[List[Coin]] = adventure.collectCoins()
>2
>3 val treasure: Try[Treasure] = coins match {
>4    case Success(cs) => adventure.buyTreasure(cs)
>5    case failure @ Failure(t) => failure
>6 }
>Basically what this is explicitly saying is that 
>adventure.collectCoins()
>may fail and the success or failure of that operation can be used in the
>next operation which is to buy treasure using those coins. At this point
>this is just expressing the semantics of the operation directly in the
>code, namely that the operation may fail (as is also the case in
>buyTreasure). This isn't a programming error, it's just expressing that
>perhaps the adventure character couldn't collect coins and/or wasn't able
>to buy treasure (maybe they didn't have enough coins). So Erlang's "Let It
>Crash" philosophy isn't applicable.  It can be argued that the block at
>lines 3-5 just a case expression, but this misses the point that the
>language directly expresses, through Try[], that both collectCoins() and
>buyTreasure can fail.

Right, so this could be:

    Coins = adventure:collect_coins(),
    Treasure = case Coins of
        {ok, Cs} -> adventure:buy_treasure(Cs);
        Error -> Error % or should it be thrown?
    end

But this does look like a regular `case` expression, and I'm guessing 
the example of composability would need more complexity to better show 
what you mean.

I am not ready to say that 'Let it crash' doesn't apply. You can very 
well let it crash there, but this is a question that depends on whether 
you decide to or not, and what you might need to report to the user.

>
>The example above is interesting from a semantic perspective, but it's
>mixing the happy path with failure handling. This is where the next
>concept/capability of Akka is interesting, namely the ability to compose
>operations while separating the failure handling path from the "happy
>path".  Here's the follow-on example from the same course:
>

This is where a difference is made with these types/monads, yes!

>1  val treasure: Try[Treasure] =
>2    adventure.collectCoins().flatMap(coins => {
>3       adventure.buyTreasure(coins)
>4    })
>5
>6  treasure match {
>7     case Success(successValue) =>
>9        do something like continue to the next challenge...
>10   case Failure(errorValue) =>
>11     do something like make the character repeat the previous challenge...
>
>So the "happy path" of collectCoins() and buyTreasure() isn't intermingled
>with what to do if one or both of these operations fail. Specifically,
>buyTreasure() won't throw an exception if collectCoins() fails. I don't
>know of any way to express this in Erlang.

Yes. So the regular `try .. catch` attempt would be:

    try
        {ok, Cs} = adventure:collect_coins(),
        Res = lists:flatmap(fun(Coins) ->
            {ok, Val} = adventure:buy_treasure(Coins),
            Val
        end, Cs),
    of
        SuccessValue ->
            %% Do something like continue to next challenge
    catch
        Type:Reason ->
            %% Do something like maybe repeat the previous challenge
    end.

Given what we care about here is whether we actually failed *anywhere* 
or *nowhere* (at least based on your failure value match), this is 
equivalent to what you have. Interestingly, because you could be 
expected to see each operation fail, you might also have a callee go for 
harder failures:

    try
        lists:flatmap(fun(Coins) -> adventure:buy_treasure(Coins) end,
                      adventure:collect_coins())
    of
        SuccessValue -> % Keep going
    catch
        _:_ -> % alt path
    end

The distinction is there, and really, the challenge is picking which one 
to implement when you design the 'adventure' module. I'd argue for the 
former if you expect multiple operations to 'fail' (it is likely that a 
character cannot connect coins without it being a programmer error), so 
that the caller can choose how to handle alternative branches at every 
level.

The cumbersome alternative is to indeed bake in the flow control to the 
values, yielding things like:

    handle_coins() ->
        case collect() of
            {ok, Val} -> % keep going
            Error -> % error path
        end.

    collect() ->
        case adventure:collect_coins() of
            {ok, Coins} -> buy_treasure(Coins);
            Error -> Error
        end.

    buy_treasure() ->
        %% and so on

It's one we do actually see a lot in Erlang overall, especially if you 
end up wanting specific handling of alternative result at each level 
(logging a message, storing metrics, returning a failed value, undoing 
operations, backoff, whatever) which could be harder to do at one end or 
the other.

>
>As far as I know Erlang doesn't support "Try" and the associated use of it
>in "match". In Akka this support is provided by monads. erlando is an
>Erlang library intended to provide monad support, but monads aren't baked
>into the language. And "Try" just applies to synchronous operations.
>"Future" implements the same semantics to async operations. "Observable"
>implements the same semantics to asynchronous stream operations.
>

Right.

>I don't have enough real-world experience in whether or not these concepts
>are encountered in day-to-day "reactive" programming in Erlang or
>Scala/Akka, but they do seem useful. This leads me to my questions:
>
>1. Are these concepts generally useful or just interesting from an academic
>perspective?

They are truly useful in my opinion, but how needed they are may depend 
on what exception handling mechanism you have. Erlang does tend to have 
that pattern made explicit with sequences of case expression (as in my 
last example). Whether this is boilerplate that ought to be eliminated 
is likely a question of personal preferences.

>2. Would it be useful to support these capabilities as first-class concepts
>in Erlang (similar to gen_servers)? Or is this so trivial in Erlang that
>it's not worth making these first class capabilities?

This is a more interesting question, because Erlang does have a lot of 
ways to handle exceptions. I mean you can have actual exceptions, option 
types, tagged values, multiple return values, signals, continuations, 
mixes of them, and so on.

Mahesh Paolini-Subramanya, in one of his Erlang factories presentations 
(see 
http://www.erlang-factory.com/conference/Chicago2013/speakers/MaheshPaoliniSubramanya), 
defined the following concerns when handling failures in order to get 
fault tolerance:

1. Separation of Concerns (Each people can be treated individually)
2. Error Encapsulation (The infection doesn't spread)
3. Fault Detection (Make sure you know someone is infected)
4. Fault Identification (You have got this specific strain of flu!)

Separation of concerns os usually given by having each process entirely 
isolated in Erlang, and tends to compose well with Error Encapsulation.

What is interesting is that error encapsulation is invariably best done 
at the lowest level of abstraction possible (this is what isolates an 
error the most from the rest of the system!); conversely, fault 
detection and identification tend to get broken by this approach -- if 
you eat up errors, you never know what they properly are. This is the 
dreaded 'try ... catch' that catches everything and returns 'ok' as if 
everything was fine.

So to me the question is not necessarily whether it's worth making a 
first-class capability out of it, but whether adding it would help in 
any of these cases.

For example, one of the very interesting form we see in a production 
proxy here is the following form:

    relay(Client, Server) ->
        case read(Client) of
            done ->
                {ok, Client, Server};
            {ok, Data} ->
                send(Client, Server, Data);
            {error, Reason} ->
                {error, upstream, Reason}
        end.

    send(Client, Server, Data) ->
        case gen_tcp:send(Server#server.port, Data) of
            ok -> relay(Client, add_metrics(Server, Data));
            {error, Reason} -> {error, downstream, Reason}
        end.

Now the actual code is more complex than this (lots more funnier cases 
with buffering and error-detection)

This kind of mechanism is nice to have because it helps in all cases:

1. Obtained through individual processes
2. An error interrupts the flow ASAP and tells you *why*
3. The error condition bubbles up as a well known tuple (and we also
   take care of identifying good terminations)
4. We know specifically why a condition failed, and whether it was due
   to the client or the server, which turns out to be critical when
   relaying the information to customers.

So for me the question is rather whether option types / Either types / 
maybe monads retain the necessary flexibility. If the semantics of the 
maybe monad end up similar, to say:

    relay(Client, Server) ->
        case read(Client) of
            done ->
                {ok, Client, Server};
            {ok, Data} ->
                ok = gen_tcp:send(Server#server.port, Data),
                relay(Client, add_metrics(Server, Data))
        end.

Meaning that we just fail and lose contextual data when it happens, then 
I, as a programmer, may not get much in terms of expressiveness outside 
of not relying on exceptions. In this case, I'd say the benefit is minor 
at best and let-it-crash (with an optional try ... catch) is still good 
enough for pretty much all intents and purposes. I get a similar result 
(happy path, exceptions/branches may lose some contextual data, apparent 
composability, etc.)