[erlang-patches] Add net_kernel and epmd support for multiple simultaneous distributed transport protocols

[Kenneth Lundin]

Hi Serge,

See embedded comments at the end.


On Thu, Jan 9, 2014 at 9:26 PM, Serge Aleynikov <serge@REDACTED> wrote:

> Kenneth,
>
> Thank you for the thorough response! Your reasoning is appealing and I'd
> like to engage in the discussion on how we can move on to include this
> functionality in the distribution, as I believe the community will benefit
> greatly from being able to run an Erlang node listening on multiple transports
> concurrently.
>
> The changes in the patch (https://github.com/erlang/otp/pull/121) cover
> the following areas:
>
> 1. Extension of the EPMD distributed protocol (here we can safely add new
> set of commands while being fully backward compatible)
>     erts/doc/src/erl_dist_protocol.xml
>
> 2. Modification of the EPMD daemon to support the new protocol version and
> to keep track of the multiple protocols/ports per node (this is also done
> in a backward compatible way. Nodes supporting new protocol version
> communicate using new spec, and old nodes communicate using old spec)
>     erts/epmd/src/epmd.h
>     erts/epmd/src/epmd.c
>
> 3. Modification of the EPMD command-line tool to talk to the local EPMD
> daemon using the new protocol version (the command-line tool uses new
> protocol version. This is reasonable, since it's installed together with
> the version of VM supporting the new EPMD protocol features)
>     erts/epmd/src/epmd_cli.c
>     erts/epmd/src/epmd_cli.c
>     erts/epmd/src/epmd_int.h
>     erts/epmd/src/epmd_srv.c
>
> 4. Updates of the UDS distribution example code. I believe the example in
> the current release is outdated (I was unable to compile it), but I
> patched it anyway to illustrate the usage of the new protocol)
>     lib/kernel/examples/uds_dist/src/dist_selector.erl
>     lib/kernel/examples/uds_dist/src/uds_dist.erl
>     lib/kernel/examples/uds_dist/src/uds_server.erl
>
> 5. Change of the net_kernel transport pluggable architecture to allow
> support of the new EPMD protocol.
> 5.1 Net_kernel support for the new protocol.
> 5.2 TCPv4 support of the new protocol
> 5.3 TCPv6 support of the new protocol
> 5.4 TLS support of the new protocol
>     lib/kernel/src/net_kernel.erl
>     lib/kernel/src/dist_util.erl
>     lib/kernel/src/inet6_tcp_dist.erl
>     lib/kernel/src/inet_tcp_dist.erl
>     lib/kernel/src/net_kernel.erl
>     lib/ssl/src/inet_tls_dist.erl
>     lib/ssl/src/ssl_tls_dist_proxy.erl
>     lib/kernel/doc/src/net_kernel.xml
>
> It seems to me that we could roll it out in two phases:
>
> 1. Implement 1,2,3 in the first phase.  This would allow to deploy a
> version with EPMD supporting old and new protocols.
> 2. Implement changes to net_kernel and transports (4,5) to be able to talk
> to new EPMDs
>
> If agreed, I can break up my patch in two, corresponding to this plan.
>
> What do you think?
>

I think it is a good idea to roll out this in 2 or more phases but I think
we need think a bit more about this before we can give a more feedback on
exactly what to include in the different phases.

Will be back with more comments/suggestions soon.

Regards Kenneth

>
> Serge
>
>
>
> On Thu, Jan 9, 2014 at 10:53 AM, Kenneth Lundin <kenneth.lundin@REDACTED>wrote:
>
>> On Fri, Jan 3, 2014 at 4:30 PM, Serge Aleynikov <serge@REDACTED>wrote:
>>
>>> I'd like to inquire the status of the following pull request:
>>> https://github.com/erlang/otp/pull/121
>>>
>>> Has it been reviewed?
>>>
>>> Thanks!
>>>
>>> Serge
>>>
>>> Hi Serge,
>>
>> We have discussed your contribution at the OTP technical board and here
>> are the results:
>>
>> Inclusion in OTP
>> -----------------------
>>
>> Your patch touches many source files which are central for the Erlang
>> distribution and we feel it would require quite a lot of work from our side
>> as well to assure that nothing in the current distribution gets broken or
>> incompatible. It would also require quite deep involvement from us giving
>> more detailed feedback regarding interfaces, semantics etc. We can not
>> prioritize this at the moment so we will not include your contribution now
>> (which means REJECT for now).
>>
>> But we really want to add this kind of functionality and want to
>> encourage you (and others) to continue the good work in this area for
>> possible inclusion in OTP at a later stage. See more info and thoughts
>> below.
>>
>>
>> General thoughts
>> -----------------------
>>
>> Enhancing/extending the Erlang distribution to support heterogenous
>> communication protocols is something we have discussed many times and it is
>> something that we really would like to have.
>>
>> 1) We would like a plugin architecture where different transport
>> protocols can be easily plugged in without having to change the basic
>> Erlang distribution or the VM.
>>
>> 2) It should be possible to run different protocols towards different
>> nodes
>>
>> 3) should be possible to have different timeouts (NET_TICK_TIME) per
>> connection
>>
>> 4) interesting protocols would be UDS, TCP, TLS over TCP, SCTP, ...
>>
>> 5) it should be possible to look up nodes in different ways and maybe not
>> only via contacting epmd on the host name which is part of the node name.
>> I.e having the hostname as part of the node name is not always what you
>> want, depending on protocol to use.
>>
>> ....
>>
>> Your specific contribution
>> -----------------------------------
>> In your pull request you have implemented fully or partly 1,2, 4, and we
>> see it as a good start.
>>
>> A general comment is that the diffs are showing differences on many
>> places in the code where you actually haven't changed anything except tabs,
>> line breaks and spaces. This makes it more difficult to sort out what you
>> have changed and added. We would like a clean diff without unnecessary
>> touches lines which has no relevance for the new function. If a formatting
>> change is needed or suggested it should be a separate commit/pull request.
>>
>> Regarding details in your code we have not had the time to take a deeper
>> look but from a quick glance it looks good. The questionmarks are more
>> about how to introduce this new functionality in a safe way and what more
>> is needed that you have not addressed so far.
>>
>>
>> Suggested way forward
>> ------------------------------
>>
>> I think that one approach that would make it easier to include this kind
>> of functionality soon would be to make minimal changes to the existing
>> distribution code and only add or change so that it is possible to support
>> heterogenous protocols with code that initially can live as a separate
>> extension that don't need to be part of the OTP distro.
>> In this way the functionality can be further developed and used together
>> with an ordinary OTP distro (unpatched) and can easily be included later
>> (hopefully within a year).
>>
>> Maybe you think that you already have done those minimal changes, but I
>> suggest you take another look and see if it is possible to find a way to
>> make minimal or now changes to net_kernel and epmd etc. which we could
>> include and then have the rest as a separate "application"/extension
>> project at github that can be included later.
>>
>> Just some thoughts:
>> Maybe let the old distribution work as before and let the user choose
>> "new_distro" in some way and will only then get to run the new code which
>> allows heterogenous protocols another new EPMD or an EPMD written in Erlang
>> or allow for plugins of different EPMD implementations that can run in
>> parallell. The communication towards EPMD maybe needs to be secure as well
>> and then it might be much easier to implement in Erlang.
>>
>> I (we) are more than willing to discuss more about how to find this
>> minimal patch which we think we can "safely" include and which makes it
>> possible to develop and use the rest as pure extensions to OTP in a first
>> step.
>>
>> Regards Kenneth, Erlang/OTP Ericsson
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20140115/2bda53fd/attachment.htm>