[erlang-patches] [PATCH] Fix SCTP multihoming for IPv6

[Tomas Abrahamsson]

On Tue, Apr 24, 2012 at 16:11, Raimo Niskanen
<raimo+erlang-patches@REDACTED> wrote:
> On Mon, Apr 23, 2012 at 11:21:08PM +0200, Tomas Abrahamsson wrote:
>> On Mon, Apr 23, 2012 at 16:16 +02:00, Raimo Niskanen wrote:
>> > Is it so that the sctp_bindx argument 2 declared as (struct sockaddr *)
>> > is in fact misused to be able to contain IPv6 addresses by packing
>> > them unaligned?
>> >
>> > If so, I would like this fact more visible in the code by defining 'addrs' as:
>> >        char addrs[256 * sizeof(struct inet_address)]
>
> But before that the Linux man page says:
>        If sd is an IPv4 socket, the addresses passed must be IPv4 addresses.
>        If sd is an IPv6 socket, the addresses passed can be either IPv4 or
>        IPv6 addresses.
>
> So it might be possible to mix them... And if that is allowed the only way
> I can see is packing them in a char array.
>
> If it is not allowed to pack them in a char array, the sctp_bindx call
> should be able to detect mixed addresses immediately on the first
> differing address.

I've done some experimenting.  It seems I was a bit too
early in claiming that multihoming works with the patch.
Should have tested other OSes.  There are several issues.

I have tested Linux, Solaris 11 (OpenIndiana 151a3)
and to some extent also FreeBSD 8.2.

On Solaris and FreeBSD, one must call bind before calling
sctp_bindx.  On Linux it is required according to the man
page, but in practice, it seems to work also with no call to
bind at all.  This means ipv4 multihoming currently does not
work on Solaris and FreeBSD, with or without any patch.
I've verified that on OpenIndiana and FreeBSD 8.2.

Additionally, FreeBSD accepts only one address at a time in
the sctp_bindx call.  Quoting the man page: "The argument
addrs is a list of addresses (where the list may be only 1
in length)".

On Solaris, mixing ipv4 addresses and ipv6 adresses in the
call to sctp_bindx is not allowed according to the man page.
Specifying ipv4-mapped ipv6 addresses is allowed, though.
But it seems to half-work in practice: sctp_bindx does not
return an error when adding an ipv4 address to an ipv6 sctp
socket, but on the other hand, it does not seem to work to
connect to it over ipv4 and send sctp messages.  The client
thinks it successfully connects, but the server does not see
anything.  Same symptoms both when specifying an ipv4
address and an ipv4-mapped ipv6 address to sctp_bindx:
can't get ipv4 sctp communication to work.

On Linux, mixing ipv4 and ipv6 addresses seems to work in
the call to sctp_bindx seems to work well.  It is possible
to connect and send messages also over ipv4 to an ipv6
socket with ipv4 addresses added to it, and even to connect
over ipv4 to an ipv6 socket with ipv4-mapped ipv6 addresses
added to it.  (Tested internally within one host, though, not
between hosts.)

So to make multihoming work also on Solaris and FreeBSD, we
would need to make it call bind before calling
sctp_bindx, and call sctp_bindx repeatedly with only one
address at a time.

About calling bind before sctp_bindx, I guess the proper
place to change would be to make gen_sctp:open call bind on
the first ip address, and sctp_bindx any following ip
addresses.  Currenty, it calls sctp_bindx (only), if more
than one ip address is specified, otherwise it calls bind.
The decision to call bind or sctp_bindx happens in
inet:open/8.

   ~ ~ ~

If we would like to go the route of allowing mixing ipv4 and
ipv6 addresses in the sctp_bindx call, then we would also
need to change the inet_drv.c and prim_inet.erl, so that the
SCTP_REQ_BINDX and INET_REQ_BIND messages also carry some
kind of indication saying if the address(es) is/are ipv4 or
ipv6 address(es).  Currently, the prim_inet.erl sends 2
bytes port number followed by 4 or 16 bytes of address, and
inet_drv.c expects 4 or 16 byte addresses, depending on the
socket's/port's family (ipv4 or ipv6).

With the patch previously submitted, specifying for instance
two ipv6 addresses and one ipv4 address to gen_sctp:open,
will cause a badarg, because inet_set_address in inet_drv.c
will not find enough bytes to unpack, thus returning einval.
Specifying 3 ipv4 addresses (3 * (2+4) = 18 bytes) for an
ipv6 sctp socket (expected number of bytes = 2+16) might not
cause an error, but unexpected surprises instead.  Ouch :(

   ~ ~ ~

Regarding patching, I suppose it ought to be split into
something like 3 commits: one fixing the bind + sctp_bindx
issue, another adding support for mixing ipv4 and ipv6
addresses and the last commit updating the preloaded
prim_inet (never done that before, so please forgive any
error in reasoning here).  The first commit would have a
value in its own, since it would add multihoming support for
non-Linux OSes, while also adding ipv6 multihoming support
for Linux.  Does this sound ok?


BRs
Tomas