1-to-1 connections & possibly hidden nodes

[Roberto Ostinelli]

This is pretty interesting Fred, thank you for sharing. I didn't know about
this possibility. Are you aware of any pitfalls of using this method? I
will investigate it too.

On Mon, Jul 12, 2021 at 10:35 PM Fred Hebert <mononcqc@REDACTED> wrote:

> Another approach that I recall working is one that is cookie-based. Each
> cookie you set represents cluster membership, so nodes that share a cookie
> will connect to each other.
>
> If you create another node that has a distinct cookie, it won't connect to
> anything else. If I recall properly, you can however call
> erlang:set_cookie(OtherNode, Cookie) to make a single caller able to use a
> distinct cookie to talk to OtherNode, meaning it will connect to it, but
> won't be able to transitively forward the connection unless the peer also
> shares that cookie.
>
> This lets you use non-hidden nodes that are parts of clusters but still
> have them able to privately talk to special nodes by using one-off cookies
> that act as a way to wall off connection propagation.
>
> On Mon, Jul 12, 2021 at 2:54 PM Stanislav Ledenev <s.ledenev@REDACTED>
> wrote:
>
>> If I understand you correctly, erl -connect_all false is what you need.
>>
>> пн, 12 июл. 2021 г. в 21:27, Roberto Ostinelli <ostinelli@REDACTED>:
>>
>>> ...And of course the diagram would be (without the copy-pasta issues):
>>>
>>> <NODE A in cluster> --------- <NODE C in cluster>
>>>         \                     /
>>>          \                   /
>>>           \                 /
>>>           <NODE B in cluster>
>>>            *       *       *
>>>           *        *        *
>>>          *         *         *
>>>       <NODE B1> <NODE B2> <NODE B3>
>>>
>>>
>>> On Mon, Jul 12, 2021 at 8:24 PM Roberto Ostinelli <ostinelli@REDACTED>
>>> wrote:
>>>
>>>> All,
>>>> Is it possible to have a 1-to-1 connection from a node in a cluster to
>>>> a single Erlang node which does _not_ connect to the mesh, but only to a
>>>> very specific Erlang node?
>>>>
>>>> I've go a very CPU-heavy application and am trying to find ways to
>>>> offload the CPU computations to Erlang nodes (tha don't need to be in a
>>>> cluster, except for a single box they refer to).
>>>>
>>>> Basically something like this:
>>>>
>>>> <NODE A in cluster> --------- <NODE B in cluster>
>>>>         \                     /
>>>>          \                   /
>>>>           \                 /
>>>>           <NODE B in cluster>
>>>>            *       *       *
>>>>           *        *        *
>>>>          *         *         *
>>>>       <NODE B1> <NODE B2> <NODE B>
>>>>
>>>> The distributed cluster is composed of nodes A, B and C, but nodes B1,
>>>> B2 and B3 can communicate directly with node B and that node only. Node B
>>>> can offload CPU computations to B1, B2, B3 by sending erlang messages to
>>>> them and receive results back.
>>>>
>>>> Every node in the cluster needs ~50x the CPU power, therefore I'm
>>>> trying to avoid having the standard cluster grow so big that it would be
>>>> generating way too much noise in a full mesh network (especially since it
>>>> really isn't needed).
>>>>
>>>> I can of course use a simple custom TCP binary channel, but was
>>>> wandering if something already exists that I could leverage. I've seen that
>>>> hidden nodes "Hidden nodes always establish hidden connections to all other
>>>> nodes except for nodes in the same global group" [1] so if I understand
>>>> this correctly nodes B1-3 would still connect to A and B, even though the
>>>> cluster wouldn't know about them.
>>>>
>>>> Thank you for any considerations you might have,
>>>> r.
>>>>
>>>>  [1] http://erlang.org/doc/man/erl.html
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20210713/8a13ea61/attachment.htm>