[erlang-questions] Sender punishment removed

[Jesper Louis Andersen]

As Lukas already wrote on slack somewhere:

Imagine you have a 2000'es machine: single core, around 400 mhz if you are
lucky. In that setting, sender punishment can in some situations rectify a
system that is going overboard. We simply let the offending process have
less time on the scheduler in the hope that the overloaded mailbox process
can catch up and do its work. It is not a surefire solution, but it may
avoid some situations in which the system would otherwise topple.

Fast forward 18 years. Now, the machines are multicore, at least 4 threads
commonly. Here, a sender might live on one core whereas the reciever might
live on another process. It is less clear why the punishment strategy is
good: we get to stop the sender, but there were already a scheduler for the
other core and it is still overloaded. Worse, perhaps all the other cores
are able to send messages through to the overloaded process.

As for the flow control: Erlang systems already employ flow control, namely
TCP flow control between distributed nodes. I've seen two recent problems
pertaining to having flow control inside flow control: gRPC has 3 layers:
gRPC itself, HTTP/2 and TCP. And HTTP/2 has a layer on top of TCP. This is
dangerous as the flow control of the underlying system can interfere with
the flow control of the system above.

By extension, any Erlang-mechanism of flow control needs to protect against
a scenario where your application has its own layer and make sure it
doesn't interfere.

Personally, I think Ulf Wiger's "Jobs" model paved the way[0]: Apply flow
control on input edge of the system, but don't apply it internally. If you
do this correct, then a system shouldn't overload because of the
border-limit. If you apply internal flow control, you also expose yourself
to the danger of an artificial internal bottleneck. Rather, sample
internally and use this as a feedback mechanism for the input edge.

Also note distributed flow control is a considerably harder problem to
solve, and since Erlang is distributed by default, any general solution has
to address this as well.

[0] https://github.com/uwiger/jobs/blob/master/doc/erlang07g-wiger.pdf

On Fri, Jan 19, 2018 at 9:51 AM Karl Nilsson <kjnilsson@REDACTED> wrote:

> So I saw that the sender punishment was removed in [1]. The commit message
> doesn't outline any of the reasoning behind this. Is there any more details
> available about this anywhere I can read? I understand it never really
> worked that well but it would still be interesting to understand a bit
> further.
>
> On a similar note what is the current thinking on flow control between
> erlang processes? Are there any improvements on mixing in a few calls in
> with the casts?
>
> [1]
> https://github.com/erlang/otp/commit/2e601a2efc19d64ed0628a5973596e6331ddcc7c
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20180119/72ca209b/attachment.htm>