[erlang-questions] Beginner trying to figure out idiomatic OTP architecture

[Garrett Smith]

Here's some detail behind syn:

http://www.ostinelli.net/an-evaluation-of-erlang-global-process-registries-meet-syn/

It's also an excellent read on how to evaluate options and then,
inevitably, build your own ;)

Just an aside to your overall approach - I tend to blanch at architectural
considerations like these as they tend to be academic, unless you *really*
understand what you need to build ahead of time. And if you *really*
understand what you're building (e.g. you've built something like it a few
times before) then a so called architecture is implicit and you have only
specific technical issues to solve.

I have an admitted tendency to be glib about architecture and I realize
nothing is ever as simple as I think it is :) But if you have a solid
understanding of OTP application "architecture" (it's really more of
configuration) you can start trail blazing without the slightest fear of
building yourself into corners. The gist of OTP is to wrap, somewhat
clumsily, Erlang processes in higher level interfaces. You get "process
management" out of that - in a similar way you might get "memory
management" out of some C/C++ framework.

A very nice characteristic of the Erlang Tao is that you build small
independent components (sure, call them micro services - they're closer to
nano) that participate in a system. If you don't like something, you tend
to add, modify or remove something small. So net splits and channels and
fancy stuff like that are discrete problems in the context of your
otherwise stable application. In my experience, you can defer solving these
things until you're in the middle of them. And then solve them. Or try to.
Then iterate because you probably not 100% happy.

I.e. you don't need to get it all down before you start.

If you have questions about canonical OTP - and this I think you should get
correct - ask here.

It's slightly under documented but e2 [1] is a solid wrapper around OTP
that does get you a canonical OTP "architecture" - but with a higher
signal-to-noise ratio in code.

[1] http://e2project.org

On Wed, Mar 23, 2016 at 11:53 PM Matthew Shapiro <me@REDACTED> wrote:

> Thank you very much,  Ranch, gproc, and syn seem like perfect frameworks
> to handle those portions of it and simplifies a lot of things (though I
> need to do some research to figure out the difference between gproc and
> syn).  So that solves those issues, well although it looks like I will have
> to jump through some hoops to get Ranch working on Windows.
>
> In regards to the channel per process vs not, I think my mind went to that
> idea due to knowing that in normal IRC servers channels have other
> particular aspects to them (such as titles and modes, etc...) and I guess
> those aspects made my mental model lean towards channels as individual
> processes (even though I admit those features aren't part of my
> requirements since this is just to get my feet wet).
>
> While I haven't gotten to clustering stuff in Erlang yet, my idea was to
> guarantee that if a netsplit occurs you can communicate with user in your
> channels that are connected to the same node as you are in.  I don't know
> yet if that changes the architecture at all but in my mind I'm not sure if
> it does (channel manager/channel processes would just relay the messages to
> the other nodes).
>
> Anyways, some pretty interesting and enlightining things to think about
>
>
> On Wed, Mar 23, 2016 at 10:19 AM, Jesper Louis Andersen <
> jesper.louis.andersen@REDACTED> wrote:
>
>>
>> On Tue, Mar 22, 2016 at 4:06 AM, Matthew Shapiro <me@REDACTED> wrote:
>>
>>> I am now at the point where I want to test some of this knowledge out,
>>> and I thought a good idea was to create a (basic) IRC server (as I've
>>> written them in the past in other languages, and it seemed like a good use
>>> case for Erlang).
>>
>>
>> Here is how I would do it:
>>
>> * There is no reason to run your own acceptor pool. Every client
>> connecting runs behind the `ranch` acceptor pool.
>>
>> * The thing that happens concurrently in an IRC server are the connecting
>> clients. There is relatively little need for a channel to act on behalf of
>> itself, so one may look at a solution where a channel is just a list of
>> members, handled by a general manager of channel lists in ETS. Posting a
>> message to a channel is simply looking up interested parties, and sending
>> the message to all of them. OTOH, having a process per channel could be
>> helpful in order to proxy messages via the channel process: send to the
>> chan process, and have it forward to the recipients. Which is best depends
>> is not a priori clear to me, but when I did this years ago, I managed to do
>> this without channel processes.
>>
>> * Consider managing the registry of Pids through either the `gproc` or
>> the `syn` frameworks. This avoids you having to redo most of the nasty
>> parts of registry and you can avoid the problems of using atoms only as in
>> the local registry.
>>
>> * If you want your server to run as a cluster, you will have to think
>> about the problem of a netsplit inside the cluster and what guarantees you
>> want to have.
>>
>> This leaves your supervisor tree in one of two forms: Either the
>> top-level supervisor runs a single channel manager process worker, or it
>> runs a simple_one_for_one pool of channels together with a manager for
>> creating/removing channels, if you deem it necessary to keep a process
>> tracking each channel.
>>
>> In general, I would avoid solutions where you "hand off" state between
>> processes as if they were sitting in a pipeline. It is often better to make
>> the process itself run as an independent system. Joe said "An Erlang web
>> server runs 2 million webservers, each serving one request." In this case,
>> you could argue you want to run a couple thousand IRC servers, each serving
>> one channel, and a couple thousand connection proxies, each serving one TCP
>> connection, connecting to multiple such channels. A connection proxy then
>> has the task of transforming the outside IRC protocol into nice symbolic
>> Erlang terms band and forth. And the Channel servers are tasked with
>> handling pub/sub between the processes, as well as ownership management of
>> the channels in question.
>>
>> The trick is to get events/messages to flow between the connection
>> processes such that the emergent behavior of a wild IRCd suddenly appears :)
>>
>> --
>> J.
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160324/d43128b5/attachment.htm>