[erlang-questions] epmd regression bug in erlang-solutions.com esl-erlang 18.3?

Mon Mar 21 10:57:45 CET 2016

On 2016/3/21 5:27, Michael Santos wrote:
> On Mon, Mar 21, 2016 at 02:23:05AM +0800, Khitai Pang wrote:
>> On 2016/3/20 6:53, Michael Santos wrote:
>>> On Sat, Mar 19, 2016 at 11:47:49PM +0800, Khitai Pang wrote:
>>>> On 2016/3/19 19:45, Michael Santos wrote:
>>>>> On Mar 18, 2016 10:46 PM, "Khitai Pang" <khitai.pang@REDACTED
>>>>> <mailto:khitai.pang@REDACTED>> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I have been using the esl-erlang pacakge from
>>>>> https://packages.erlang-solutions.com/erlang/ Ubuntu PPA for
>>>>> Trusty.
>>>>>> I have these in my bashrc:
>>>>>>
>>>>>> export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX"   (LAN IP of my VM in
>>>>> the cloud)
>>>>>> export ERL_EPMD_PORT="5678"
>>>>> Just tested and it worked for me.
>>>>>
>>>>> Does the IP address in ERL_EPMD_ADDRESS exist in the VM?
>>>>>
>>>> Yes it does, it's the IP of the LAN in the clound, and it all works
>>>> fine with 18.2.
>>>>
>>>> I believe it's possible that the issue is caused by the
>>>> configuration in the operating system (Ubuntu 14.04.4 x86_64) but
>>>> currently I have no idea how to further debug this issue.
>>>>
>>>> With esl-erlang 18.2 and the following configs in bashrc:
>>>>
>>>> export ERL_EPMD_ADDRESS="XXX.XXX.XXX.XXX"
>>>> export ERL_EPMD_PORT="5678"
>>>>
>>>> epmd works fine:
>>>>
>>>> tcp        0      0 127.0.0.1:5678          0.0.0.0:* LISTEN      6754/epmd
>>>> tcp        0      0 10.131.70.80:5678       0.0.0.0:* LISTEN      6754/epmd
>>>>
>>>>
>>>> Thanks
>>>> Khitai
>>>>
>>>>>> Everything worked fine with the 18.2 package.  After upgrading
>>>>> to esl-erlang 18.3, epmd stopped to work:
>>>>>>> epmd -debug
>>>>>> epmd: Sat Mar 19 16:16:51 2016: epmd running - daemon = 0
>>>>>> epmd: Sat Mar 19 16:16:51 2016: failed to bind socket: Cannot
>>>>> assign requested address
>>>>>> There is a workaround: if I remove the ERL_EPMD_ADDRESS
>>>>> environment variable, epms works:
>>>>>> tcp        0      0 0.0.0.0:5678 <http://0.0.0.0:5678> 0.0.0.0:*
>>>>> LISTEN      3328/epmd
>>>>>> tcp6       0      0 :::5678 :::*                    LISTEN     3328/epmd
>>>>>> After downgrading esl-erlang from 18.3 to 18.2, the issue is gone.
>>> Have you disabled IPv6?
>>>
>>>      # disable IPv6 support on the loopback device
>>>      $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=1
>>>
>>>      $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd
>>>      epmd: Sat Mar 19 22:40:09 2016: failed to bind socket: Cannot assign requested address
>>>
>>>      # enable IPv6 support on the loopback device
>>>      $ sudo sysctl net.ipv6.conf.lo.disable_ipv6=0
>>>
>>>      $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 epmd
>>>      <works>
>>>
>>> If we look at what is happening:
>>>
>>>      $ ERL_EPMD_ADDRESS="127.0.0.1" ERL_EPMD_PORT=5678 strace epmd
>>>      ...
>>>      bind(4, {sa_family=AF_INET6, sin6_port=htons(5678), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EADDRNOTAVAIL (Cannot assign requested address)
>>>      ...
>>>
>>> When you specify a specific address to bind, epmd will also bind to the
>>> IPv4 and IPv6 loopbacks. In this case, your OS supports IPv6 but has
>>> disabled it on the loopback device so the bind fails:
>>>
>>>      $ ifconfig lo
>>>      lo        Link encap:Local Loopback
>>>                inet addr:127.0.0.1  Mask:255.0.0.0
>>>                UP LOOPBACK RUNNING  MTU:65536  Metric:1
>>>                RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>                TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>                collisions:0 txqueuelen:0
>>>                RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>>
>>> So either enable IPv6 for the loopback or disable IPv6 completely by
>>> passing "ipv6.disable=1" at boot.
>> Michael,
>>
>> Thanks for the workaround.  2 questions:
>>
>> 1)  I have these in my sysctl:
>>
>> net.ipv6.conf.all.disable_ipv6 = 1
>> net.ipv6.conf.default.disable_ipv6 = 1
>> net.ipv6.conf.eth0.disable_ipv6 = 1
>> net.ipv6.conf.eth1.disable_ipv6 = 1
>> net.ipv6.conf.lo.disable_ipv6 = 1
>>
>> Aren't these enough to tell epmd that ipv6 is not available on loopback?
> We can see from your first email that allocating an IPv6 socket and
> binding it to the IPv6 ANY address works:
>
> tcp        0      0 0.0.0.0:5678 0.0.0.0:* LISTEN      3328/epmd
> tcp6       0      0 :::5678      :::*      LISTEN      3328/epmd
>
> A system with a working IPv6 implementation must have an IPv6 loopback.
> So I would say your system configuration is broken. If the intent was to
> disable IPv6, it didn't work: you have a half-working system.

Agreed.

> I do think epmd should require explicitly specifying all addresses to
> bind instead of implicitly binding the loopback. So in your case you
> would need to add the IPv4 loopback but it would otherwise work as
> before:
>
>      $ ERL_EPMD_ADDRESS="10.131.70.80,127.0.0.1" ERL_EPMD_PORT=5678 epmd

So I should either disable IPv6 completely or enable IPv6 on loopback?

>> 2)  Why does 18.2 work in the same environment?
> 18.3 supports node registration for distribution over IPv6 (OTP-13364):
>
> http://erlang.org/download/otp_src_18.3.readme

I see, thanks for the information.

Thanks
Khitai