epmd

[Richard Barrett]

I raised the problem on this mailing list of multiple copies of epmd 
being spawned when multiple copies erl were run, each of which only 
knew about the instance of beam that spawned it. Further, all epmd 
instances were successfully listening on port 4369.

Initial opinions in response were that this should be impossible and 
might be due to a screwed up kernel on my Suse 6.2 Linux controlled 
machine.

I have investigated further and believe that all is working as 
advertised with Suse Linux and the problem with empd arises from a 
conditional compilation decision in the distribution source file 
otp_src_R6B-0/erts/empd/src/epmd_srv.c

Around lines 135 to 146 of this source file, conditional compilation 
of a setsocketopt call to enable the SO_REUSEADDR option occurs if 
the OS being compiled under is not _WIN32_.

In effect, the coding assumption is that on non-_WIN32_ (that is, on 
UNIX or Linux I guess), an attempt by a new instance of epmd to bind 
to port 4369 when another instance of epmd is already bound to that 
port, will fail even if the SO_REUSEADDR socket option is set on the 
socket being used to bind to the port  This bind failure is in effect 
used to determine if epmd is already running.

It turns out that Suse 6.2 Linux (as on _WIN32_) and possibly other 
versions of Linux, take the SO_REUSEADDR socket option at face value. 
In consequence, bind failure, when this socket option is used, is 
inadequate as the sole determinant of whether epmd is already extant 
in the system.

Certainly, commenting out the relevant conditionally compiled lines 
in epmd_srv.c and rebuilding epmd has solved the multiple epmd 
instances problem I had.

For reasons I have yet to determine, under Sun Solaris, an attempt to 
bind to a given port by a second instance of my test scripts (see 
below) does fail, even though the SO_REUSEADDR socket option has be 
enabled. So, Solaris and Suse 6.2 Linux appear to interpret the 
semantics of the SO_REUSEADDR socket option rather differently and 
I've no opinion as to which is correct.

Incidentally, I explored this problem using some short Python 
scripts, copies of which are given below. Thanks to those readers who 
contributed a response to my original posting.

--------------------------------------------------------
Experimental Python scripts are as follows.

server script enabling SO_REUSEADDR --------------------

from socket import *
import os
pid = os.getpid()
HOST = ''
PORT = 4369
s = socket(AF_INET, SOCK_STREAM)
s.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
reuse = s.getsockopt(SOL_SOCKET, SO_REUSEADDR)
print "Server process pid=%d starting with SO_REUSEADDR=%d" % (pid, reuse)
s.bind(HOST, PORT)
s.listen(1)
conn, addr = s.accept()
s.close()
print "Process %d connected to %s with SO_REUSEADDR=%d" % (pid, addr, reuse)
while 1:
	data = conn.recv(1024)
	if not data: break
	data = data + (", returned by pid=%d" % pid)
	conn.send(data)
conn.close()

server script not enabling SO_REUSEADDR --------------------

from socket import *
import os
pid = os.getpid()
HOST = ''
PORT = 4369
s = socket(AF_INET, SOCK_STREAM)
reuse = s.getsockopt(SOL_SOCKET, SO_REUSEADDR)
print "Server process pid=%d starting with SO_REUSEADDR=%d" % (pid, reuse)
s.bind(HOST, PORT)
s.listen(1)
conn, addr = s.accept()
s.close()
print "Process %d connected to %s with SO_REUSEADDR=%d" % (pid, addr, reuse)
while 1:
	data = conn.recv(1024)
	if not data: break
	data = data + (", returned by pid=%d" % pid)
	conn.send(data)
conn.close()

client script --------------------
from socket import *
import os
pid = os.getpid()
print "Client process pid=%d starting" % pid
HOST = ''
PORT = 4369
s = socket(AF_INET, SOCK_STREAM)
s.connect(HOST, PORT)
s.send("Hello, world sent from pid=%d" % pid)
data = s.recv(1024)
s.close()
print 'Received', `data`
--------------------------------------------------------------
------------------------------------------------------------------
Richard Barrett, PostPoint 27,         e-mail:r.barrett@REDACTED
Fujitsu Telecommunications Europe Ltd,      tel: (44) 121 717 6337
Solihull Parkway, Birmingham Business Park, B37 7YU, England
"Democracy is two wolves and a lamb voting on what to have for
lunch. Liberty is a well armed lamb contesting the vote."
Benjamin Franklin, 1759
------------------------------------------------------------------