The Erlang way - dynamic upgrade of a server and UBF extensions

[Joe Armstrong]

On Tue, 29 Apr 2003, Jay Nelson wrote:

> I like clear, clean, simple, regular data structures.  Especially
> in a functional language they make for obvious code.  One
> thing I've always felt was missing in data protocols is the
> versioning on every packet.  I think that is an essential feature.
>

  <aside>

  Yes  - everybody  alwas forgets  the version  number  - *everything*
should always have  a version number.  The trouble  is when you create
something you think it is perfect and will never need upgrading - this
is always false.

Erlang modules should begin:

	-need_version(2.3).

Or something.

They got this right in XML :-) and PDF

</aside>

  Yes - Actually  I woke up in  the middle of the night  (as one does)
and thought about this and the packet length problem.

  It might not have been clear in my last mail that the version was the
version number of the contract not of the protocol.

  I think we need a prefix like this:

  +------+-----------+-------------------+	
  | UBFv | Contractv | ..................|
  +------+-----------+-------------------+

  UBFv      is the version of the UBF protocol itself
  Contractv is the version of the contract (ie the data)

 
> The IP failover seems to handle a reasonable case of
> essentially connectionless single packet transmission,
> or the equivalent of frequency hopping.  More complex
> failover can be part of a data packet listing multiple IPs
> or a hierarchy or mapping to particular versions of the
> protocol, as agreed in UBF(B) or UBF(C) contracts.
> 

Interesting - perhaps the whole of client-server RPC is wrong.

Should it be cluster-cluster RPC.

The client could enclose a list of IPs and the server could reply to any
client in the list.

  On *every* reply the server would reply with a list of servers to be
used on  the next  call - one  the next  call the client  could choose
*any* server.

(This will not be wildly efficient - because of globally replicate the
state in the clusters at either end - *but* it will be easy to
program fault-tolerent applications)

> I like the idea of always having a packet header.   It is much
> simpler if the size of the packet is specified, but I wonder if
> that is too much of a burden on the client.  I would rather:
> 
> <<Length:32>> = gen_tcp:recv(Socket, 4),
> Packet = gen_tcp:recv(Socket, Length) ...
> 

At first I thought - you're right and then I thought (later) you're wrong.

In UBF if you get any pack of any size you just add it to the input queue,
then you parse it byte-at-a-time until you hit $ then pop the recognition
stack.

If you want the length first you can't just output the data structure
as it is generated - which made me think you need yet another type
for streaming media.

That would need yet another operator.

Lists are sent as

# a & b & c & d & ...

So I guess infinite streams are

* a ! b ! c !  ...

Or something.

I also though more about the versioning problem - synchronous change 
in the middle of a session would be very difficult to program and understand.

So how about:

When A and B communicate

A knows that it can speak versions 1,2,3,4,8,12,20  of a contract
B knows that is can speak versions 1,2,3,9,12,20,32 of a contract

At the start A & B negotiate the highest common protocol
this is version 20 - and they proceed.

A gets a SW exception at some point (a bug) - it fails and removes
20 from its "available contracts" list.

The session is restarted from the beginning - this time
A can speak versions 1,2,3,4,8,12 and so a retry is
done at level 12 (not 20).

  Both  sides therefor  degrade to  the  level where  they think  they
understand the protocol. (And A's error log has an entry to tell the
programmer why level 20 failed - a contract violation).

 This seems simple enough that it might work.


/Joe

> then to parse a stream a few bytes at a time calling recv.
> 
> 
> Trying to come up with a header runs into three problems:
> 
> 1) If packet size is first, 4 bytes might be wasteful to a small
> msg protocol.
> 
> 2) If version comes first to determine the size of the packet
> length field, either a fixed version size (too limiting) is needed
> or the integer UBF(A) spec is used with byte by byte recv parsing.
> 
> 3) Having a size at the front means the client has to assemble
> the whole message, compute the size and then send all, rather
> than streaming out data.
> 
> Not sure who should have the easier job, the client or server.
> Given that the client is written more often (and probably by a
> wider range of programmers of varying skill level) I would guess
> it is better to make things easy on the client.  That would argue
> for a header as Joe proposed without a length and the version
> coming first (presumably an integer in UBF(A) format).  The
> server has to do byte by byte gen_tcp:recv in this scenario.
> 
> jay
>