[eeps] Signals

[Richard A. O'Keefe]

On 27/09/2014, at 7:28 PM, Marcin Sokolowski wrote:

> In many high-demand systems control flow and data use separate (often physically separate) communication channels so that even in case of overflow in data channel system is still responsive for control type of messages. Erlang does not have such built-in mechanism.

Erlang has two related mechanisms.

(1) Having two processes in the façade.  Send urgent messages
    to one, non-urgent to the other.  Resolve it internally
    however you please.  One approach here is to have the
    "control" process supervise the "data" process.

(2) Selective receive.

    This is the usual answer to the problem.

> == Problems it addresses ==
> 
> 
> 1) if process is processing loads of messages there is no efficient and elegant way of controlling this process as control messages would queue up behind "data" messages. Selective receive is not the answer as it cost grows with the size of the queue.

Having a queue whose size grows much is a problem whether you
have urgent messages or not.

Since messages are received only when a process feels like
receiving them, nothing could stop a process always looking
in its normal mailbox before looking in the urgent mailbox,
thus making urgent messages queue up *anyway*.

> I propose following syntax for sending signals
> 
> 
> Pid ! Message % to send normal priority message
> 
> Pid !! Signal % to send high priority, control messages

Joe Armstrong has already defined !! to mean something
quite different.
> 
> erlang:signal(Pid, Signal).
> 
> == Receiving ==
> 
> Receiving signal would be no different to receiving normal message. Signals should appear as if they are added to the head of the Inbox (unless of course other signals are already present)

This feels entirely wrong.  For one thing, it is way too coarse-
grained.  You want "we're running out of petrol" to overtake
"Are we there yet?" but leave no room for "the engine's on fire!"
to overtake "we're running out of petrol".  For another, it
creates the anomaly that the "priority" of a message is determined
by something other than its contents.  You could have one 'ping'
message going to the end of the mailbox and another identical
'ping' message going to the front of the mailbox.  And perhaps I
am too much influenced by OO ideas, but it just seems wrong for
something other than the process itself to be in charge of mailbox
sorting.  If urgency really matters, this approach is a bug waiting
to happen, because Pid!Urgent will often work just as well as
Pid!!Urgent, and using the more familiar form is *way* too easy a
mistake to make.  Oh, and if you have a process that doesn't
*expect* urgent messages, having two ways to send messages could
break ordering properties it was relying on.

I claim that

 (1) The priority of a message should be decided by the receiving
     process, not the sending process; and
 (2) it should be determined exclusively by the form of the message
     (so that you *can't* send it with the wrong priority).

For example, a scheme where a process could do

    process_flag(urgent, atom_or_nil) -> atom_or_nil

    with
        process_flag(urgent, [])
    meaning the present behaviour, and
        process_flag(urgent, 'mayday')
    meaning that messages of the form {'mayday',....} would
    overtake messages of other forms,

would at least be less bug-prone.
> 
> For this mechanism to work as expected across the network signals should have their dedicated TCP connections.

As soon as you send a message through TCP you have no guarantee
of urgency nohow.  
What does adding a second TCP connection do?
Nothing!
There is no way to say "this connection should be expedited
relative to that one".  In particular, there is no reason to
expect the "signal" connection to experience lower delays
than the "plain" one.  The only way to get guaranteed order
for messages is to send them on the same connection, but
then, to quote the Wikipedia page,

	TCP OOB data was not designed for the modern Internet.
	The urgent pointer only alters the processing on the
	remote host and
>>>	doesn't expedite any processing on the network itself.
	When it gets to the remote host there are two slightly
	different interpretations of the protocol, which means
>>>	only single bytes of OOB data are reliable.
	This is assuming it is reliable at all as it is one of
	the least commonly used protocol elements and tends to
	be poorly implemented.

A single SCTP connection might be more useful than two TCP ones.

> 
> 
> 2) Any operational type of messages that are sent to control behaviour of the process should be sent as signals. For processes handling large number of messages it is not possible to control process quickly enough - control message might arrive way to late.

The proposed change to Erlang will not fix that.

> 
> 1) requiring processes to acquire "special permission" to send signals, without it signals would be just simple messages

OUCH.  So you have a special form to get critically needed
special behaviour, except it might not act that way after all.
It would be far better for signal sending to be a checked run
time error when not supported.

In any case, the "abuse" issue here is that a process might
legitimately send "urgent" messages to *some* processes but
not others.  Making it all-or-none is way too crude.

> 2) making sending signal blocking operation that is semantically very different to asynchronous message sending

Why?
> 
> 3) limiting the type of messages that could be sent as signals (e.g. atoms only)

There does not seem to be any safety improvement in that restriction,
but it would cripple the mechanism to the point where it is doubtful
that it would be much use.

> I do not know much about Erlang VM internals but 2 possible solutions come to my mind:
> 
> 1) Separate inboxes: one for messsages, one for signals.
> 
> 2) Double-headed inbox queue, one head for messages, one for signals.

It looks as though it could be done with a single extra pointer
and maybe a dummy node.

What we are missing here is a demonstration that Erlang actually
*needs* this feature.  There are some things that keep popping up
in the Erlang mailing list, but this is not one of them.

It seems to me that since overfull mailboxes cause all sorts of
problems, and that selective receive works fine for "urgent"
messages when mailboxes are *not* overfull, solving the overfull
mailbox issue would be more useful.