To OTP or not to OTP (Was a Joeish distribution)

[Francesco Cesarini]

I would be *very* careful to go out and say what you did when the 
majority of non ericsson people on this list have learnt Erlang on their 
own, and are developing systems without the proper mentorship of people 
who have used Erlang on a large scale.

OTP was built based on many years of project experiences, feedback from 
users, and extensive prototyping. The design principles which came out 
of it fill a gap which exists in pure Erlang. The best example that 
comes to my mind is people asking "How can I apply OO design" in Erlang? 
The reason is that they are trying to fill in this gap.

The gap covers

* Structuring programs & general SW architecture issues
* Reusability
* Maintenance
* Fault Tolerance
* Design Patterns

I would agree that there are some pretty ugly inconsistencies and 
constructs in some of the principles. But they are minor, and often just 
syntactical compared to the advantages you get from using them.

I work with people who have learnt Erlang on their own, with out the 
proper mentorship, on a daily basis. They do not have the time to 
develop their own theories on how to structure and develop large scale 
SW development using Erlang. There is no need for them to reinvent the 
wheel. They need to get products out, and they need to get them out as 
fast as possible. Using OTP and the libraries' well tested code is today 
the only way to do it efficiently.

You guys developed OTP. Developed Erlang. Have massive experience and 
knowledge and know from the start what is right, what is wrong, 
alongside all the special cases to handle. What you do not seem to 
understand is that that is not the case out there. The 50% code 
reduction we usually achieve when migrating a non OTP system to OTP says 
it all. And that covers just the code size, not the added functionality.


Francesco
--
http://www.erlang-consulting.com

Joakim G. wrote:

> Hi,
> I have thought about Francesco's questions about how to structure an
> Erlang system with/without OTP.
> 
> I would avoid OTP at least for a middle sized software projects
> involving 5-35 programmers.
> 
> Why? I would like to keep full transparency to the use of primitive
> Erlang constructs, e.g. (async/sync) message passing, receive
> statements, loop based servers, EXIT signal propagation etc.
> 
> OTP's high level abstraction and callback oriented approach alienates
> the programmer from what actually is going on in the system. This is
> bad. It also extremely boring. If you want Xt programming why not go
> for that directly.
> 
> In larger project (>35) I might use OTP or maybe even better: Don't
> start the project at all. Too big. I would prefer to divide the larger
> project into *very* (as in *very*) separate middle sized projects
> instead. You know all that. If not possible. Do something else.
> 
> This is how I would go about it:
> 
> Compile a short best practice guideline on how two write three types
> of servers, i.e. gen_serv.erl, tcp_serv.erl, app_serv.erl. I would
> keep the majority of the guidelines as in-line comments in the code
> skeletons for these servers.
> 
> gen_serv
>     Loop based, receive based, message passed based, nice init
>     phase, clear API.
> {tcp,udp}_serv
>     s`Same characteristics as for gen_serv above but for tcp and
>     udp.
> app_serv
>     Supervision of gen_serv and {tcp,udp}_serv instances.
> 
> (I will not delve deeper here).
> 
> Each sub-group within the larger project would be responsible for its
> app_serv instance and all started processes should adhere to the
> mechanisms "enforced" by gen_serv and tcp_serv skeletons.
> 
> Not that far from what OTP tries to give! Only *much* more light
> weight and without sacrificing Erlang transparency. No "fun" removed.
> 
> I would setup two global services (servers) as well to be used by all
> sub-groups within the project:
> 
> 1. Distributed and persistent configuration registry (ala Window's
>    Registry) with support for set, get, and validation rules for
>    values as well as subscription support for value changes. Nice
>    export/dump format etc.
> 2. An extensible log server. The error_logger module comes pretty close
>    with OTP contamination and all. I may rewrite it.
> 
> That concludes my "pure" Erlang approach to OTP.
> 
> All the above might be possible to describe in a single chapter in the
> new Erlang book (code skeletons and all).
> 
> Cheers
> /Jocke
> 
> Klacke wrote:
> 
>>> You would not divide the processes into behaviors? 
>>
>>
>>
>> Nope
>>
>>
>>> You would not place your behaviors in supervision trees? 
>>
>>
>>
>>
>> Nope
>>
>>
>>> You would not encapsulate your supervision trees in applications? 
>>
>>
>>
>> Nope
>>
>>
>>> You would not glue together your applications and start them using a 
>>> boot file? 
>>
>>
>>
>> Nope
>>
>>
>>> If you use the generic behaviors or not, that is another story The 
>>> OTP design principles are something beginners *should* start doing as 
>>> soon as they are confident with Erlang, as it is the natural step 
>>> towards writing their own.
>>
>>
>>
>>
>> Sorry, don't agree.
>>
>>
>>> If you prefer to avoid the systools, 
>>
>>
>>
>>
>> Yes,
>>
>>
>>> or implement your own application controller as the existing one 
>>> lacks many features, fine. If you stay away from the release handler, 
>>
>>
>>
>> Absolutely.
>>
>>
>>
>>> you are not alone (I avoid it myself whenever possible). And if you 
>>> use your own error logging mechanism, that makes two of us. But I 
>>> have a hard time following how you can not build a large coherent 
>>> system without using the theory behind the Erlang/OTP design 
>>> principles as it is described in the documentation manuals or in the 
>>> OTP course material we have been using on thousands of people for 
>>> almost 8 years. I would be very careful to recommending people 
>>> evaluating Erlang or building their first product to bypass the 
>>> principles, as that is what makes Erlang even more powerful than what 
>>> it already is.
>>>
>>>
>>>
>>>> Not me anyway, I don't like them (because I don't understand them)
>>>
>>>
>>>
>>> I am sure we can invite you when we give a course near you ;-)
>>
>>
>>
>> :-)
>>
>>
>> I'll tell you what I use. I use the gen_servers. I have never used any
>> of the other gen_ modules   (except gen_tcp of cource ... but that
>> is an entirely different beast, then gen_ in gen_tcp only has the
>> name in common with gen_fsm and gen_event)
>>
>> I actually do use applications. Not that I like them but if I want to
>> write an erlang program that fits into the otp tree I must do that.
>>
>> I've never used the release handler, I barely understand the 
>> error_logger. Everytime I need to do something with the error_logger such
>> a duplicating error_messages to go to yet another place I have to look 
>> deep and hard at the code again.
>>
>> I've never had a supervision tree deeper than 1.
>>
>>
>> /klacke
> 
> 
> 
> 
>