[erlang-questions] Beginner trying to figure out idiomatic OTP architecture

[Matthew Shapiro]

I guess I need to read more on how clustering actually works in an OTP way
then as I'm not totally sure how I would handle that without at least one
dedicated process (even if it's one for all channels that just manages the
members list).  I figured the clustering would come into play by having
either a channel manager process or dedicated channel process for each
channel connected to on a specific node, then have the channel
manager/channel process relay that out to the other nodes.  Otherwise I"m
not clear on how a user process would send a message to all users in a
channel because it would have to have some way to find out what user
processes it would have to send the message to without having to broadcast
it out to every user process and let them decide if they should process it
or not.

Unless the solution to that is Mnesia to manage channel details (haven't
gotten that far in my Erlang learning yet).

On Thu, Mar 24, 2016 at 11:36 PM, Park, Sungjin <jinni.park@REDACTED>
wrote:

>
>
> On Thu, Mar 24, 2016 at 11:02 AM, Matthew Shapiro <me@REDACTED> wrote:
>
>> Thank you very much,  Ranch, gproc, and syn seem like perfect frameworks
>> to handle those portions of it and simplifies a lot of things (though I
>> need to do some research to figure out the difference between gproc and
>> syn).  So that solves those issues, well although it looks like I will have
>> to jump through some hoops to get Ranch working on Windows.
>>
>> In regards to the channel per process vs not, I think my mind went to
>> that idea due to knowing that in normal IRC servers channels have other
>> particular aspects to them (such as titles and modes, etc...) and I guess
>> those aspects made my mental model lean towards channels as individual
>> processes (even though I admit those features aren't part of my
>> requirements since this is just to get my feet wet).
>>
>
> This may be too early to point but I think you'd be better avoid
> dedicating a process to handle a channel.  When it comes to scaling things
> out, there would eventually be clients connected to different servers than
> the channel process.  Then imagine what, all the messages should travel to
> the process and back, which is too inefficient obviously.  Better design
> things in distributed fashion.
>
>
>> While I haven't gotten to clustering stuff in Erlang yet, my idea was to
>> guarantee that if a netsplit occurs you can communicate with user in your
>> channels that are connected to the same node as you are in.  I don't know
>> yet if that changes the architecture at all but in my mind I'm not sure if
>> it does (channel manager/channel processes would just relay the messages to
>> the other nodes).
>>
>> Anyways, some pretty interesting and enlightining things to think about
>>
>>
>> On Wed, Mar 23, 2016 at 10:19 AM, Jesper Louis Andersen <
>> jesper.louis.andersen@REDACTED> wrote:
>>
>>>
>>> On Tue, Mar 22, 2016 at 4:06 AM, Matthew Shapiro <me@REDACTED>
>>> wrote:
>>>
>>>> I am now at the point where I want to test some of this knowledge out,
>>>> and I thought a good idea was to create a (basic) IRC server (as I've
>>>> written them in the past in other languages, and it seemed like a good use
>>>> case for Erlang).
>>>
>>>
>>> Here is how I would do it:
>>>
>>> * There is no reason to run your own acceptor pool. Every client
>>> connecting runs behind the `ranch` acceptor pool.
>>>
>>> * The thing that happens concurrently in an IRC server are the
>>> connecting clients. There is relatively little need for a channel to act on
>>> behalf of itself, so one may look at a solution where a channel is just a
>>> list of members, handled by a general manager of channel lists in ETS.
>>> Posting a message to a channel is simply looking up interested parties, and
>>> sending the message to all of them. OTOH, having a process per channel
>>> could be helpful in order to proxy messages via the channel process: send
>>> to the chan process, and have it forward to the recipients. Which is best
>>> depends is not a priori clear to me, but when I did this years ago, I
>>> managed to do this without channel processes.
>>>
>>> * Consider managing the registry of Pids through either the `gproc` or
>>> the `syn` frameworks. This avoids you having to redo most of the nasty
>>> parts of registry and you can avoid the problems of using atoms only as in
>>> the local registry.
>>>
>>> * If you want your server to run as a cluster, you will have to think
>>> about the problem of a netsplit inside the cluster and what guarantees you
>>> want to have.
>>>
>>> This leaves your supervisor tree in one of two forms: Either the
>>> top-level supervisor runs a single channel manager process worker, or it
>>> runs a simple_one_for_one pool of channels together with a manager for
>>> creating/removing channels, if you deem it necessary to keep a process
>>> tracking each channel.
>>>
>>> In general, I would avoid solutions where you "hand off" state between
>>> processes as if they were sitting in a pipeline. It is often better to make
>>> the process itself run as an independent system. Joe said "An Erlang web
>>> server runs 2 million webservers, each serving one request." In this case,
>>> you could argue you want to run a couple thousand IRC servers, each serving
>>> one channel, and a couple thousand connection proxies, each serving one TCP
>>> connection, connecting to multiple such channels. A connection proxy then
>>> has the task of transforming the outside IRC protocol into nice symbolic
>>> Erlang terms band and forth. And the Channel servers are tasked with
>>> handling pub/sub between the processes, as well as ownership management of
>>> the channels in question.
>>>
>>> The trick is to get events/messages to flow between the connection
>>> processes such that the emergent behavior of a wild IRCd suddenly appears :)
>>>
>>> --
>>> J.
>>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
>
> --
> Park, Sungjin
>
> -------------------------------------------------------------------------------------------------------------------
> Peculiar travel suggestions are dancing lessons from god.
>   -- The Books of Bokonon
>
> -------------------------------------------------------------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160325/9565693f/attachment.htm>