[erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3?

[Michael Santos]

On Mon, Mar 21, 2016 at 02:23:05AM +0800, Khitai Pang wrote:
> On 2016/3/20 6:53, Michael Santos wrote:
> >On Sat, Mar 19, 2016 at 11:47:49PM +0800, Khitai Pang wrote:
> >>On 2016/3/19 19:45, Michael Santos wrote:
> >>>On Mar 18, 2016 10:46 PM, "Khitai Pang" <khitai.pang@REDACTED
> >>><mailto:khitai.pang@REDACTED>> wrote:
> >>>>Hi,
> >>>>
> >>>>I have been using the esl-erlang pacakge from
> >>>https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for
> >>>Trusty.
> >>>>I have these in my bashrc:
> >>>>
> >>>>export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX"   (LAN IP of my VM in
> >>>the cloud)
> >>>>export ERL_EPMD_PORT="5678"
> >>>Just tested and it worked for me.
> >>>
> >>>Does the IP address in ERL_EPMD_ADDRESS exist in the VM?
> >>>
> >>Yes it does, it's the IP of the LAN in the clound, and it all works
> >>fine with 18.2.
> >>
> >>I believe it's possible that the issue is caused by the
> >>configuration in the operating system (Ubuntu 14.04.4 x86_64) but
> >>currently I have no idea how to further debug this issue.
> >>
> >>With esl-erlang 18.2 and the following configs in bashrc:
> >>
> >>export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX"
> >>export ERL_EPMD_PORT="5678"
> >>
> >>epmd works fine:
> >>
> >>tcp        0      0 127.0.0.1:5678          0.0.0.0:* LISTEN      6754/epmd
> >>tcp        0      0 10.131.70.80:5678       0.0.0.0:* LISTEN      6754/epmd
> >>
> >>
> >>Thanks
> >>Khitai
> >>
> >>>>Everything worked fine with the 18.2 package.  After upgrading
> >>>to esl-erlang 18.3, epmd stopped to work:
> >>>>>epmd -debug
> >>>>epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0
> >>>>epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot
> >>>assign requested address
> >>>>There is a workaround: if I remove the ERL_EPMD_ADDRESS
> >>>environment variable, epms works:
> >>>>tcp        0      0 0.0.0.0:5678 <http://0.0.0.0:5678> 0.0.0.0:*
> >>>LISTEN      3328/epmd
> >>>>tcp6       0      0 :::5678 :::*                    LISTEN     3328/epmd

> >>>>
> >>>>After downgrading esl-erlang from 18.3 to 18.2, the issue is gone.
> >Have you disabled IPv6?
> >
> >     # disable IPv6 support on the loopback device
> >     $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=1
> >
> >     $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd
> >     epmd: Sat Mar 19 22:40:09 2016: failed to bind socket: Cannot assign requested address
> >
> >     # enable IPv6 support on the loopback device
> >     $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=0
> >
> >     $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd
> >     <works>
> >
> >If we look at what is happening:
> >
> >     $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 strace epmd
> >     ...
> >     bind(4, {sa_family=AF_INET6, sin6_port=htons(5678), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EADDRNOTAVAIL (Cannot assign requested address)
> >     ...
> >
> >When you specify a specific address to bind, epmd will also bind to the
> >IPv4 and IPv6 loopbacks. In this case, your OS supports IPv6 but has
> >disabled it on the loopback device so the bind fails:
> >
> >     $ ifconfig lo
> >     lo        Link encap:Local Loopback
> >               inet addr:127.0.0.1  Mask:255.0.0.0
> >               UP LOOPBACK RUNNING  MTU:65536  Metric:1
> >               RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >               TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >               collisions:0 txqueuelen:0
> >               RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
> >
> >So either enable IPv6 for the loopback or disable IPv6 completely by
> >passing "ipv6.disable=1" at boot.
> 
> Michael,
> 
> Thanks for the workaround.  2 questions:
> 
> 1)  I have these in my sysctl:
> 
> net.ipv6.conf.all.disable_ipv6 = 1
> net.ipv6.conf.default.disable_ipv6 = 1
> net.ipv6.conf.eth0.disable_ipv6 = 1
> net.ipv6.conf.eth1.disable_ipv6 = 1
> net.ipv6.conf.lo.disable_ipv6 = 1
> 
> Aren't these enough to tell epmd that ipv6 is not available on loopback?

We can see from your first email that allocating an IPv6 socket and
binding it to the IPv6 ANY address works:

tcp        0      0 0.0.0.0:5678 0.0.0.0:* LISTEN      3328/epmd
tcp6       0      0 :::5678      :::*      LISTEN      3328/epmd

A system with a working IPv6 implementation must have an IPv6 loopback.
So I would say your system configuration is broken. If the intent was to
disable IPv6, it didn't work: you have a half-working system.

I do think epmd should require explicitly specifying all addresses to
bind instead of implicitly binding the loopback. So in your case you
would need to add the IPv4 loopback but it would otherwise work as
before:

    $ ERL_EPMD_ADDRESS="10.131.70.80,127.0.0.1" ERL_EPMD_PORT=5678 epmd

> 2)  Why does 18.2 work in the same environment?

18.3 supports node registration for distribution over IPv6 (OTP-13364):

http://erlang.org/download/otp_src_18.3.readme

> Thanks
> Khitai