[erlang-questions] UDP questions

[zxq9]

Hi Joe.

tl;dr: Problems with UDP boil down to limitations of the protocol.

On 2016年1月27日 水曜日 16:27:13 Joe Armstrong wrote:
> Hello,
> 
> I have a simple UDP client/server
> 
> The server is on a fixed port 4567 - I open it like this
> 
>     {ok, Socket} = gen_udp:open(4567, [binary]),
> 
> then wait for registrations
>     receive
> {udp, Socket, _Ip, _Port, <<"register">>) ->
> 
> 
> Clients do this:
> 
>      {ok, Socket} = gen_udp:open(0, [binary]),
>      gen_udp:send(Socket, "localhost", 44567, <<"register">>),
>      ...
> 
> I'm testing on "localhost"
> 
> After registration the processes send binaries to each other
> 
> This works fine for small binaries but gen_udp:send fails  with
> {error,emsgsize} for large binaries
> 
> So I have a few questions:
> 
>     1) Can I find out the max binary size that can be sent/received?
>     2) Can I increase the max size?
>     3) Is there a guaranteed minimum packet length that will not be fragemented?
>     3) Can received packets be fragmented - TCP has a  {packet,N}
>        option and some built-in defragmentation, what's the story for
>        UDP?
> 
> I think I know the answers to these but I'm not really sure - could somebody
> who really knows enlighten me?

The max datagram size is supposed to be discoverable with GET_MAXSIZES() as
per RFC 1122 4.1.4 ( https://tools.ietf.org/html/rfc1122#page-79 ) but in
practice I've never seen anyone actually query this. Discovery, at least in
the game server world, tends to happen using ICMP because the question is
almost never "what is the max datagram size only to localhost".

Generally speaking you've got 512 - 8 bytes of safe space to use in IPv4
UDP -- though over the localhost... I'm not really sure what the limit would
be; but you should be able to configure the local interface to be some
almost arbitrarily large jumboframe size.

512 - 8 is a guideline based on MTU based fragmentation. I'm assuming you
want the code to not care whether its all localhost or if the system is
extended across the network. ?

As far as breaking larger things up into several pieces, you either wind
up reinventing TCP streams in chunks, or reinventing SCTP sequential packet
mode. UDP will never guarantee the ordering or receipt of any given message.
I've never lost a UDP packet to localhost, but I'm pretty sure you could
create a system load that would cause this to happen.

I think really the ideal protocol is SCTP. Each association in sequential
packet mode is like a UDP datagram, but ordered and guaranteed to be
received (deals with the fragmentation and ordering problem for you),
and any association can be "peeled off" into a stream socket connection
when you need a stream of bytes (like for transferring a large binary
blob). Also, a single connection can be one-to-many, and each connection
can have several channels -- thus also solving the problem of needing
several bands.

But only geeks even know about SCTP and its not supported everywhere, so
that might put a dent in the plan; I assume this is in the context of the
blog posts you just put up about stitching together program components
over UDP sockets.

You're either going to need multiple channels (UDP for your control messages
and TCP connections for large transfer), or a multi-channel connection
protocol like SCTP to fully accomplish what you're trying to do.

This approach, incidentally, is my preferred way of thinking about system
design -- and it makes me so sad that people think HTTP is the only protocol
that exists! (And JSON and XML are the only serialization formats!)

People just don't remember how to program sockets today -- and we don't make
it better by ignoring the problem since it is so simple in Erlang, but can
suck so hard and confusing in many other languages. Its pretty obvious why
more programs don't support things like IPv6... nobody wants to touch the
networking code if they can avoid it. (Much easier to just make an "API" 
over HTTP and pray that one day the web will be magically secure, bandwidth
will be infinite and FavoriteWebServer will support feature X that you
really wish you could use -- in other words, easier to stick one's head in
the sand.)

-Craig