[erlang-questions] Message send guarantees

[Raimo Niskanen]

On Wed, Jan 25, 2017 at 11:06:05PM +0000, Dmitry Kakurin wrote:
> Thanks Lukas, please see a couple of questions/comments below.
> 
> - Dmitry
> 
> ________________________________
> From: Lukas Larsson <lukas@REDACTED>
> Sent: Wednesday, January 25, 2017 12:37 AM
> 
> Hello,
> 
> On Wed, Jan 25, 2017 at 12:38 AM, Dmitry Kakurin <dima_kakurin@REDACTED<mailto:dima_kakurin@REDACTED>> wrote:
> When I execute "pid ! msg" and it returns, what are the guarantees if pid and self are on the same node?
> Can I assume that by the time it returns, msg is in the pid's queue? And as a result all messages sent to pid afterwards will be queued after msg?
> 
> No you cannot assume that. The message is considered "in transit" until the receiving process inspects it somehow.
> 
> What aspect of implementation can cause such reordering? Could you give an example how this could happen? Maybe I'll be able to artificially create conditions where it would not happen. Or work around it in some other way. Please see below why.

On a multi-threade system it is very hard to know in which order things
happen, so there is no "reordering" since there is no temporal order
to begin with.

Using the non-SMP virtual machine and erlang:yield/0 could do the trick,
but depending on the computer you might loose boatloads of performance.

It is also _very_ bad programming practice to pretend that asynchronous
message passing is not asynchronous.  It is solving a problem the
wrong way...

> 
> 
> The reason I'm asking is because I need a guarantee of proper message ordering in the following scenario when all processes involved are local to a node:
> 1. A --msg1--> C (process A sends msg1 to process C)
> 2. A --msg2--> B
> 3. B --msg2--> C (process B simply proxies msg2 from A to C)
> 
> I need a guarantee that msg2 will always appear in C's queue after msg1.
> 
> You have to do this using messages or some other sync mechanism.
> 
> You are right, in general it's easy to devise a simple scheme for that.
> Unfortunately I have a limitation. In my case A talks to C thru proxy B and would like to short-circuit the communication by getting rid of B and start talking to C directly. Process A needs to do that while messages thru B could be in flight. The limitation is that C is not aware of this proxy business, and cannot be changed (it's a regular GenServer that is not written by me).
> With lack of cooperation from C I cannot think of a communication protocol to accomplish what I want.

If B can be modified as needed; you can get an ack from B so both A and B
know about the bypass e.g to delay bypass until no messages are in flight,
provided that the messages from B to C are acked.

Or, maybe you can only use messages where the order does not matter.

But without modifying your requirements as you state them I see no solution.

Why do you really need to bypass some messages and still keep message
ordering - it is impossible!

> 
> Is there a way for processes A and B to observe C's queue in any way and derive some information about it to help me with my problem?

From a process on the same node as C it is possible to inspect C's queue 
through erlang:process_info/2, but then you have started to dig a deep hole
for yourself.  You will get whole queue of C and have to traverse all of it
to see if the message you sent is there.  But just to inspect the queue will
hurt message passing performance due to the locking involved.

We have had customers that have used something like that in an attemt to do
overload protection, only resulting in bad high load behaviour...

> 
> 
> P.S. I've read the FAQ 10.8 and 10.9 :-)
> 
> Please add to the FAQ if you think that you can make it clearer on this point. https://github.com/matthiasl/Erlang-FAQ
> 
> Lukas
> 


-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB