[erlang-patches] Added net_kernel support for multiple transport protocols

[Serge Aleynikov]

Fixed failing documentation formatting error, added test cases. Please
refetch:

git fetch git://github.com/saleyn/otp.git proto_dist

https://github.com/saleyn/otp/compare/erlang:master...proto_dist
https://github.com/saleyn/otp/compare/erlang:maint...proto_dist.patch


On Fri, Nov 1, 2013 at 12:58 AM, Serge Aleynikov <serge@REDACTED>wrote:

> Hi,
>
> [Note: this patch is submitted by email as as a pull request #121]
>
> Short problem description
> =========================
>   I want an Erlang node to listen on incoming connections using
>   TCPv4, TCPv6, UDS, and TLS transports on different ports.
>
>   I'd like other nodes to connect to this node using TLS if nodes
>   are in external networks, use TCPv4 or TCPv6 if nodes are
>   in the same subnet, and use UDS if nodes are on the same machine.
>
>   Present implementation requires that all nodes in the network
>   use the same transport, which is a huge limitation
>
> git fetch git://github.com/saleyn/otp.git proto_dist
>
> https://github.com/saleyn/otp/compare/erlang:master...proto_dist
> https://github.com/saleyn/otp/compare/erlang:maint...proto_dist.patch
>
> Long solution description
> =========================
>
> This patch addresses the limitation of registering only one Erlang
> distributed transport with Erlang Port Mapping Daemon, and therefore
> enabling a node to listen for distributed node connections on multiple
> port using different protocols, and selectively choose the protocol
> to use when connecting to a node from different secure and unsecure
> networks.
>
> The legacy implementation of Erlang distributed transport allows to
> start a node with more than one transport via the
> "-proto_dist mod1 mod2 ..." command-line option. But epmd didn't support
> registration of multiple protocols. And net_kernel doesn't implement a
> way of figuring out which transports are supported on a remote node and
> doesn't allow to select a desired transport protocol of choice to use it
> for connecting to the node.
>
> The current patch extends epmd by having it store the name of the
> protocol handling module passed to the -proto_dist startup option during
> node registration.  Multiple registrations are allowed, and they can be
> registered to different ports.  This way a node can start distributed
> transport listeners (e.g. TCP, SSL, UDS), and register them with epmd.
>
> When another node does epmd "PORT_PLEASE" lookup in order to connect to
> node X, it now gives to epmd a list of transport names it supports (i.e.
> the ones that were passed to it via -proto_dist option at startup)
> together with the node name it queries.  The epmd filters out only the
> transports supported by the calling client node, and gives back only
> those along with the corresponding port numbers.
>
> net_kernel now supports a new command-line option
> "-proto_dist_mf Mod Fun" which is a user-define callback function that
> can be used to resolve which transport protocol to use to connect to
> a node. This function is called after the epmd query returned a list
> of protocols supported by the target node. By default the first
> transport in the list is used. An example implementation of this
> function is included in
> `lib/kernel/examples/uds_dist/src/dist_selector.erl`.
>
> In order to support backward compatibility the EPMD protocol was
> extended with new commands documented in erl_dist_rotocol.xml. Old
> nodes can query the new EPMD version, but new nodes (using this patch)
> require new version of EPMD.
>
> Here's an illustration:
>
>   # Node 's' is started with TCP and TLS support
>   # see http://www.erlang.org/doc/apps/ssl/ssl_distribution.html#id61752
>   # note that lib/kernel/examples/uds_dist/src/dist_selector.erl
>   # module is used to do custom transport selection
>
>   $ erl -boot start_ssl -proto_dist inet_tcp inet_tls -ssl_difile
> "/home/serge/tmp/key.pem" -ssl_dist_opt server_certfile
> "/home/serge/tmp/certificate.pem" -ssl_dist_opt
> server_secure_renegotiate true client_secure_renegotiate true
> -proto_dist_mf dist_selector select -pa
> "lib/kernel/examples/uds_dist/ebin" -sname s@REDACTED
>
>   (s@REDACTED)1> erl_epmd:names().
>   [{ok,{"s",[{49645,"inet_tcp"},{41477,"inet_tls"}]}}]
>   (s@REDACTED)2>
>
> As we see the node registered itself with epmd and has TCP listener on
> port 49645, and TLS listener on port 41477.
>
> We now start another node with just TCP transport:
>
>   $ install/bin/erl -sname k@REDACTED
>   Erlang R17A (erts-5.11) [source-7e9f18c] [64-bit] [smp:4:4]
>    [async-threads:10] [hipe] [kernel-poll:false]
>
>   Eshell V5.11  (abort with ^G)
>   (k@REDACTED)1> erl_epmd:names().
>   [{ok,{"k",[{38558,"inet_tcp"}]}},
>    {ok,{"s",[{49645,"inet_tcp"},{41477,"inet_tls"}]}}]
>   (k@REDACTED)2>
>
> Connect it to node 's':
>
>   (k@REDACTED)2> net_kernel:connect_node(s@REDACTED).
>   true
>   (k@REDACTED)4> nodes().
>   [s@REDACTED]
>
> Start the 3rd node 'm' with TCP and SSL, and connect it to node 's':
>
>   $ erl -boot start_ssl -proto_dist inet_tcp inet_tls -ssl_difile
> "/home/serge/tmp/key.pem" -ssl_dist_opt server_certfile
> "/home/serge/tmp/certificate.pem" -ssl_dist_opt
> server_secure_renegotiate true client_secure_renegotiate true
> -proto_dist_mf dist_selector select -pa
> "/home/serge/tmp/otp/lib/kernel/examples/uds_dist/ebin" -sname m@REDACTED
>
>   % Let's query the EPMD to see which transports/ports are available
>   % to connect
>   (m@REDACTED)1>
> erl_epmd:port_please(s,"localhost",net_kernel:dist_protos(),infinity).
>   {ports,[{49645,"inet_tcp"},{41477,"inet_tls"}],
>          5,
>          [{epmd,{{127,0,0,1},4369}}]}
>
>   % Let the magic begin (since this is a node in the same subnet,
>   % dist_selector will select the TCP protocol
>
>   (m@REDACTED)2> net_kernel:connect_node(s@REDACTED).
>   true
>
> This is a long email, and a sizable patch.  I hope others find it useful
> and it can be included in the distribution.
>
> Regards,
>
> Serge
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20131104/f54eaf50/attachment.htm>