[erlang-questions] Message send guarantees

[Per Hedeland]

On 2017-01-26 14:09, Fred Hebert wrote:
> On 01/25, Dmitry Kakurin wrote:
>>
>> The reason I'm asking is because I need a guarantee of proper message ordering in the following scenario when all processes involved are local to a node:
>> 1. A --msg1--> C (process A sends msg1 to process C)
>> 2. A --msg2--> B
>> 3. B --msg2--> C (process B simply proxies msg2 from A to C)
>>
>> I need a guarantee that msg2 will always appear in C's queue after msg1.
>>
>> [...]
>>
>> Unfortunately I have a limitation. In my case A talks to C thru proxy B and would like to short-circuit the communication by getting rid of B and start talking to C directly. Process A needs to do
>> that while messages thru B could be in flight. The limitation is that C is not aware of this proxy business, and cannot be changed (it's a regular GenServer that is not written by me).
>> With lack of cooperation from C I cannot think of a communication protocol to accomplish what I want.
>>
>> Is there a way for processes A and B to observe C's queue in any way and derive some information about it to help me with my problem?
>>
> 
> As others have mentioned, this is not really possible. You *could* observe the queue, but it is not practical.
> 
> The interesting thing with Erlang there is that a lot of these design decisions work the same whether the processes are local or remote. The problem you see here of message ordering is very similar to
> the one you get of packet ordering when delivering data over a network.
> 
> Generally, the way to solve the problem is to label every message you send with a sequence number that increments. On every message received, the final receiver can then look at the sequence numbers
> and see if any message is missing. It can then block, ask for a retransmission, or disregard duplicated messages.
> 
> Because Erlang's guarantees for message passing only involve any given pair of processes (A -> B, or A -> C) but not more, we get a case where the following race conditions are very possible:
> 
>  A            B           C
>  | --- 1 ---> |           |
>  | --- 2 ---------------> |
>  |            | --- 1 --> |
> 
> The stop at B means that through scheduling, the message can be delayed for unkown periods of time and the delays can mess up the ordering.
> 
> The only surefire way of getting the messages to C in order is to either
> a) send them directly from a single sender every time, or b) to let C know if the messages are not in order so it can do the reordering itself.

Well, there is also (already hinted at I think) c) have A receive an ack
from C for message 1 before it sends message 2. Doing this for every
message may be an unacceptable throughput limitation in some cases, in
some other cases it is what happens "anyway" due to the nature of the
communication. In this particular case, "A talks to C thru proxy B and
would like to short-circuit the communication by getting rid of B and
start talking to C directly", it would be sufficient that A got an ack
for only the *last* via-B-message (could be a special "let's skip the
proxy" message) before it started sending directly to C. Seems like a
clean solution to me.

--Per