Hierarchical Distribution

[Rickard Green]

Hidden nodes and hidden global groups will be included in the next
r7b01 patch (which will be released soon).

The following is cut from the coming manual page for erl(1):

--8<--
  -hidden 
      Starts the Erlang system as a hidden node if the system is run
      as a distributed node. Hidden nodes always establish hidden
      connections to all other nodes except for nodes in the same
      global group. Hidden connections aren't published on neither
      of the connected nodes, i.e. neither of the connected nodes
      are part of the result from nodes/0 on the other node. See also
      hidden global groups, global_group(3). 
-->8--

and global_group(3):

--8<--
  The global_groups-key in the .config file defines the global groups:

  {global_groups, [GroupTuple]} 

  Types: 

      GroupTuple = {GroupName, [Node]}
                   | {GroupName, PublishType, [Node]} 
      GroupName = atom() (naming a global group) 
      PublishType = normal | hidden 
      Node = atom() (naming a node) 

  A GroupTuple without PublishType is the same as a GroupTuple with
  PublishType equal to normal. 
-->8--

--8<--
  A hidden node will establish hidden connections to nodes not part
  of the same global group, and normal (visible) connections to nodes
  part of the same global group. Hidden connections aren't published
  on neither of the connected nodes, i.e. neither of the connected
  nodes are part of the result from nodes/0 on the other node. 

  In a hidden global group (a global group defined with PublishType
  equal to hidden) all nodes are hidden nodes. 

  Hidden nodes can also be part of normal global groups. Nodes
  started with the -hidden switch will be hidden nodes even if they
  are part of a normal group, see erl(1). Other nodes in the group
  will not be affected by this. 
-->8--

/Rickard Green, Ericsson UAB, Erlang/OTP



Martin Bjorklund wrote:
> 
> "Vance Shipley" <vances@REDACTED> wrote:
> 
> > The logical topology uses a 'main processor cluster' (MPC).  The
> > BPs are hierarchically under a single MP while the SPs are hierarchically
> > under a single BP.
> >
> >      +----------------------------+
> >      | MPC       +----+           |      +----+       +----+
> >      |           | MP |           |------| BP |  -----| SP |
> >      |           +----+          /|      +----+ /     +----+
> >      |   +----+          +----+ / |      +----+/      +----+
> >      |   | MP |          | MP |<--|------| BP |-------| SP |
> >      |   +----+          +----+ \ |      +----+\      +----+
> >      |                           \|      +----+ \     +----+
> >      |               +----+       |------| BP |  -----| SP |
> >      |               | MP |       |      +----+       +----+
> >      |               +----+       |
> >      +----------------------------+
> >
> > So in distribution terms BPs only know about a single MP, and possibly
> > some SPs.  Each MP knows about all other MPs but only knows about it's
> > own BPs, and I guess not any SPs.
> >
> > Can the standard net_kernel be configured in this way?
> 
> This is how our system works at bluetail, in order to make it
> scalable.  We have a main cluster (which we call master cluster) of
> 2-4 master nodes.  This is normal erlang distribution; fully connected
> net, the mnesia tables are replicated on these nodes etc.  Then we
> have up to 252 slave nodes (like the BPs in the pic above).  They are
> so called 'hidden nodes'.  The reason for doing this is scalability;
> we really don't want a fully connected net of 256 nodes with mnesia
> actively replicating everything...
> 
> Now, the net_kernel in R7B cannot handle 'hidden nodes'.  It does
> handle 'hidden connections' though.  A hidden node is a distributed
> erlang node which won't show up in the nodes() list.  But it's got a
> node name, and you can use normal erlang rpc and so on.
> 
> Hidden nodes is a very powerful concept (we also use them for several
> other things), which I think should be part of the standard release.
> I'll include the modified files below.
> 
> Start a hidden node like this:
> 
>    erl -sname xxx -kernel hidden true
> 
> A hidden node only accepts hidden connection attempts.  This means
> that a hidden node can connect to another hidden node, like SP to BP
> in the picture above.  Normally, the hidden node connects to the
> normal node, although a normal node can do a 'hidden connect' in order
> to connect to a hidden node.
> 
> Try this by starting a hidden node, and a normal node.  Do a net:ping()
> from the hidden node to the normal node.  Do net_kernel:i() to print
> the connections in both nodes.  Also check erlang:nodes().
> 
> /martin
> 
>   ------------------------------------------------------------------------
>                     Name: dist_util.erl
>    dist_util.erl    Type: Plain Text (Text/Plain)
>                 Encoding: 7bit
> 
>                      Name: net_kernel.erl
>    net_kernel.erl    Type: Plain Text (Text/Plain)
>                  Encoding: 7bit