Q regarding multicast and n-way connection in a cluster

[Chandrashekhar Mullaparthi]

Right now I dont think Erlang has explicit support for receiving on a
multicast address. Sending to a multicast address is no problem though. But
you can use either ports or a linked in driver which implements that for
you. 

You can "configure" a node so that it doesn't automatically connect to all
nodes it sees. See the parameter dist_auto_connect in the kernel manual.

I could be wrong, but I think inter-node communications are multiplexed
through a single-socket-per-node pair. 

There is a heartbeat between connected erlang nodes. Check the net_ticktime
parameter in the kernel manual. When a process dies  - all processes linked
to it get the message {'EXIT', Pid, Reason} regardless of which nodes these
processes reside on. If this is because of a broken TCP connection - the
message will be {'EXIT', Pid, noconnection}. You can use node(Pid) to
determine from which node this message originated.

hth
Chandru

> -----Original Message-----
> From:	Reto Kramer [mailto:kramer@REDACTED]
> Sent:	26 February 2001 21:31
> To:	erlang-questions@REDACTED
> Subject:	Q regarding multicast and n-way connection in a cluster
> 
> I am relatively new to really getting into Erlang as opposed to just
> reading every available article about it ;-).
> 
> So far I used to be involved with building clustered (business) solutions
> that have employed a technique that seems not to be exactly in line with
> the Erlang way of architecting such systems.
> 
> I'd appreciate if someone could illustrate for me how an Erlang architect
> would build a "clustered stateful service".  Let's assume that  each
> client has a little bit of state (some kB) and that there could be many
> thousand concurrent clients hitting the cluster. Because of the large
> workload that the large number of clients generates, the cluster might
> have to get rather large (e.g. 10 to 50 machines). Also we do not want to
> pass the state back and forth between the client and the server but only
> pass a session identifier (e.g. as part of a http cookie).
> 
> For cluster membership I am used to use heartbeats over IP multicast,
> where each cluster member uses the same multicast address to publish the
> heartbeat information.  Heartbeats can carry all sorts of data (e.g. a
> servers current load, etc.). Also each server packs its "views" of who
> else is in the cluster into the heartbeat that is seen by everybody else.
> This avoids the need for a centralized "member registry".
> 
> Q1: how could I elegantly access the socket layer to send/listen on a
> multicast address?
> 
> It seemed to me that a cluster of n Erlang nodes will "implicitly" n-way
> interconnect if I have to use message sends from each server to each other
> server to maintain the cluster membership protocol. While this seems
> feasible for smaller clusters, I would get nervous about scalability if
> the cluster contains maybe 100s of machines (I would "waist" those socket
> resources for something that I could use 1 multicast socket for).
> 
> Q2: where can I find information as to how the various Erlang constructs
> (e.g. process links across nodes) translate to "OS resource usage".  I am
> specifically interested in whether inter node communication is multiplexed
> through a single socket per node-pair (or is there a socket per process or
> even per send)?
> 
> Q3: are "process links across nodes" implemented by detecting the breaking
> of a TCP connection or is there a heartbeat/timeout involved for the
> point2point connection as well?
> 
> Thanks for you help and pointer to "implementation level documentation"!
> 
> cheers,
> - Reto
> 
> 
> 
> 
NOTICE AND DISCLAIMER:
This email (including attachments) is confidential.  If you have received
this email in error please notify the sender immediately and delete this
email from your system without copying or disseminating it or placing any
reliance upon its contents.  We cannot accept liability for any breaches of
confidence arising through use of email.  Any opinions expressed in this
email (including attachments) are those of the author and do not necessarily
reflect our opinions.  We will not accept responsibility for any commitments
made by our employees outside the scope of our business.  We do not warrant
the accuracy or completeness of such information.