[erlang-questions] Message send guarantees

Fred Hebert <>
Thu Jan 26 14:09:41 CET 2017


On 01/25, Dmitry Kakurin wrote:
>
>The reason I'm asking is because I need a guarantee of proper message ordering in the following scenario when all processes involved are local to a node:
>1. A --msg1--> C (process A sends msg1 to process C)
>2. A --msg2--> B
>3. B --msg2--> C (process B simply proxies msg2 from A to C)
>
>I need a guarantee that msg2 will always appear in C's queue after msg1.
>
> [...]
>
>Unfortunately I have a limitation. In my case A talks to C thru proxy B 
>and would like to short-circuit the communication by getting rid of B 
>and start talking to C directly. Process A needs to do that while 
>messages thru B could be in flight. The limitation is that C is not 
>aware of this proxy business, and cannot be changed (it's a regular 
>GenServer that is not written by me).
>With lack of cooperation from C I cannot think of a communication protocol to accomplish what I want.
>
>Is there a way for processes A and B to observe C's queue in any way and derive some information about it to help me with my problem?
>

As others have mentioned, this is not really possible. You *could* 
observe the queue, but it is not practical.

The interesting thing with Erlang there is that a lot of these design 
decisions work the same whether the processes are local or remote. The 
problem you see here of message ordering is very similar to the one you 
get of packet ordering when delivering data over a network.

Generally, the way to solve the problem is to label every message you 
send with a sequence number that increments. On every message received, 
the final receiver can then look at the sequence numbers and see if any 
message is missing. It can then block, ask for a retransmission, or 
disregard duplicated messages.

Because Erlang's guarantees for message passing only involve any given 
pair of processes (A -> B, or A -> C) but not more, we get a case where 
the following race conditions are very possible:

  A            B           C
  | --- 1 ---> |           |
  | --- 2 ---------------> |
  |            | --- 1 --> |

The stop at B means that through scheduling, the message can be delayed 
for unkown periods of time and the delays can mess up the ordering.

The only surefire way of getting the messages to C in order is to either
a) send them directly from a single sender every time, or b) to let C 
know if the messages are not in order so it can do the reordering 
itself.

Anything else will be very hacky (implementing locks in userland and 
synchronizing everything) and would usually be more of a code smell than 
anything.

Note: The usage of sequential number is a more specific implementation 
of causal ordering, where a logical 'stamp' is used to denote a 
'happens-before' relationship between events. Vector clocks, version 
vectors, and interval tree clocks are other data structures that let you 
get more general ordering guarantees, but that much power isn't 
necessarily required here.


More information about the erlang-questions mailing list