[erlang-questions] Beginner trying to figure out idiomatic OTP architecture
Thu Mar 24 16:44:15 CET 2016
I can promise you putting a lot of effort at this level on the architecture
definition isn't usual (not to say I don't put thought into architecture
before coding, but usually not to this level of planning and discussion
The main purpose of this thread is that I picked a project I've done before
and using it as a launch pad on how to write Erlang applications in an
idiomatic way and gather the solid understanding of OTP application
architecture/configuration in the first place outside of just reading :).
I understand the domain pieces of the application, I just wasn't sure if I
was putting them together in a solid OTP way.
I actually came across that Syn article in my googling last night, and it
was a good read. That e2 framework has been bookmarked too, as it looks
quite interesting and the documentation and tutorials on that site look
extremely well written!
I would have started the project already, just have been caught up with a
lack of free time, but I can actually see how Erlang and OTP can actually
make this project much easier than in the languages I've written an IRC
server in before. It's actually gotten me pretty excited to code this up
in a day or two (looking forward to the weekend) :).
I actually think I"m going to forego ranch, Syn, and E2 for the first
iteration and implement them by hand (since this is a learning exercise
more than anything) and there are plenty of guides around to accomplish
each of those individually. Then I can start pulling pieces out and
implementing them via frameworks one by one.
On Thu, Mar 24, 2016 at 10:51 AM, Garrett Smith <> wrote:
> Here's some detail behind syn:
> It's also an excellent read on how to evaluate options and then,
> inevitably, build your own ;)
> Just an aside to your overall approach - I tend to blanch at architectural
> considerations like these as they tend to be academic, unless you *really*
> understand what you need to build ahead of time. And if you *really*
> understand what you're building (e.g. you've built something like it a few
> times before) then a so called architecture is implicit and you have only
> specific technical issues to solve.
> I have an admitted tendency to be glib about architecture and I realize
> nothing is ever as simple as I think it is :) But if you have a solid
> understanding of OTP application "architecture" (it's really more of
> configuration) you can start trail blazing without the slightest fear of
> building yourself into corners. The gist of OTP is to wrap, somewhat
> clumsily, Erlang processes in higher level interfaces. You get "process
> management" out of that - in a similar way you might get "memory
> management" out of some C/C++ framework.
> A very nice characteristic of the Erlang Tao is that you build small
> independent components (sure, call them micro services - they're closer to
> nano) that participate in a system. If you don't like something, you tend
> to add, modify or remove something small. So net splits and channels and
> fancy stuff like that are discrete problems in the context of your
> otherwise stable application. In my experience, you can defer solving these
> things until you're in the middle of them. And then solve them. Or try to.
> Then iterate because you probably not 100% happy.
> I.e. you don't need to get it all down before you start.
> If you have questions about canonical OTP - and this I think you should
> get correct - ask here.
> It's slightly under documented but e2  is a solid wrapper around OTP
> that does get you a canonical OTP "architecture" - but with a higher
> signal-to-noise ratio in code.
>  http://e2project.org
> On Wed, Mar 23, 2016 at 11:53 PM Matthew Shapiro <> wrote:
>> Thank you very much, Ranch, gproc, and syn seem like perfect frameworks
>> to handle those portions of it and simplifies a lot of things (though I
>> need to do some research to figure out the difference between gproc and
>> syn). So that solves those issues, well although it looks like I will have
>> to jump through some hoops to get Ranch working on Windows.
>> In regards to the channel per process vs not, I think my mind went to
>> that idea due to knowing that in normal IRC servers channels have other
>> particular aspects to them (such as titles and modes, etc...) and I guess
>> those aspects made my mental model lean towards channels as individual
>> processes (even though I admit those features aren't part of my
>> requirements since this is just to get my feet wet).
>> While I haven't gotten to clustering stuff in Erlang yet, my idea was to
>> guarantee that if a netsplit occurs you can communicate with user in your
>> channels that are connected to the same node as you are in. I don't know
>> yet if that changes the architecture at all but in my mind I'm not sure if
>> it does (channel manager/channel processes would just relay the messages to
>> the other nodes).
>> Anyways, some pretty interesting and enlightining things to think about
>> On Wed, Mar 23, 2016 at 10:19 AM, Jesper Louis Andersen <
>> > wrote:
>>> On Tue, Mar 22, 2016 at 4:06 AM, Matthew Shapiro <>
>>>> I am now at the point where I want to test some of this knowledge out,
>>>> and I thought a good idea was to create a (basic) IRC server (as I've
>>>> written them in the past in other languages, and it seemed like a good use
>>>> case for Erlang).
>>> Here is how I would do it:
>>> * There is no reason to run your own acceptor pool. Every client
>>> connecting runs behind the `ranch` acceptor pool.
>>> * The thing that happens concurrently in an IRC server are the
>>> connecting clients. There is relatively little need for a channel to act on
>>> behalf of itself, so one may look at a solution where a channel is just a
>>> list of members, handled by a general manager of channel lists in ETS.
>>> Posting a message to a channel is simply looking up interested parties, and
>>> sending the message to all of them. OTOH, having a process per channel
>>> could be helpful in order to proxy messages via the channel process: send
>>> to the chan process, and have it forward to the recipients. Which is best
>>> depends is not a priori clear to me, but when I did this years ago, I
>>> managed to do this without channel processes.
>>> * Consider managing the registry of Pids through either the `gproc` or
>>> the `syn` frameworks. This avoids you having to redo most of the nasty
>>> parts of registry and you can avoid the problems of using atoms only as in
>>> the local registry.
>>> * If you want your server to run as a cluster, you will have to think
>>> about the problem of a netsplit inside the cluster and what guarantees you
>>> want to have.
>>> This leaves your supervisor tree in one of two forms: Either the
>>> top-level supervisor runs a single channel manager process worker, or it
>>> runs a simple_one_for_one pool of channels together with a manager for
>>> creating/removing channels, if you deem it necessary to keep a process
>>> tracking each channel.
>>> In general, I would avoid solutions where you "hand off" state between
>>> processes as if they were sitting in a pipeline. It is often better to make
>>> the process itself run as an independent system. Joe said "An Erlang web
>>> server runs 2 million webservers, each serving one request." In this case,
>>> you could argue you want to run a couple thousand IRC servers, each serving
>>> one channel, and a couple thousand connection proxies, each serving one TCP
>>> connection, connecting to multiple such channels. A connection proxy then
>>> has the task of transforming the outside IRC protocol into nice symbolic
>>> Erlang terms band and forth. And the Channel servers are tasked with
>>> handling pub/sub between the processes, as well as ownership management of
>>> the channels in question.
>>> The trick is to get events/messages to flow between the connection
>>> processes such that the emergent behavior of a wild IRCd suddenly appears :)
>> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions