[erlang-questions] non-trivial supervisor idioms? -- An open-source plea!

Geoffrey Biggs geoffrey.biggs@REDACTED
Wed Oct 27 11:05:59 CEST 2010 

I'd rather see an Erlang Cookbook. The Perl and Python ones are wonderful.

Geoff


On 27/10/10 18:03, Edmond Begumisa wrote:
> I suppose you're right.
> 
> Perhaps there's another solution. My problem is this... if you do a
> google search for "hex encode in Erlang" you'll see a large number of
> really bad code snippets. Yet the authors present these as solutions. I
> for one used one of these 'solutions' for a while when just starting out
> with Erlang. As I got more proficient, one day I looked back at that
> code and thought "that's horrible" (Erlang is unique this way in that it
> can take you a really long time to learn how to recognise bad code.)
> Now, multiply and this by all the functions required to write an
> application and what you end up with is some not-so-good code released
> as open-source that other programmers relatively new to Erlang look to
> as a 'how-to' reference. This cannot be good for the language as a whole.
> 
> Maybe Ericsson could oversee the development of a new open source tool.
> I would personally love to see an Asterisk in Erlang, but I have purely
> selfish reasons.
> 
> - Edmond -
> 
> 
> On Wed, 27 Oct 2010 19:17:43 +1100, Ulf Wiger
> <ulf.wiger@REDACTED> wrote:
> 
>>
>> Edmond,
>>
>> One big problem for a company releasing large and complex,
>> battle-tested software
>> as Open Source is that Ericsson - with a ~30% share of the global
>> mobile infrastructure
>> market - is a juicy IPR target. Since it's entirely possible - nay,
>> almost a certainty - that
>> code of that size will infringe on some existing patents*, and given
>> that it is a practical
>> impossibility to figure out which (unless you happen to be the owner
>> of the patent,
>> and know what to look for), it is unlikely to happen.
>>
>> * most of which would probably not hold up in court, but challenging
>> them in court
>>   is still a risky and expensive process - not least with huge
>> opportunity cost.
>>
>> There are a few feeble protections against being sued for patent
>> infringement:
>>
>> - Use a fringe technology. This invalidates many claims, as they are
>> often fairly
>>   language-specific.
>> - Do not search the patent database. This may seem absurd, but you
>> probably
>>   won't find anything anyway, but if you did search, you may
>> eventually get sued
>>   for wilful infringement, which will increase the penalty significantly.
>> - Do not release your source code, so that it can be indexed and
>> subjected to
>>   automated search programs by patent raiders.
>> - Stay small and uninteresting, so no one will bother suing you.
>> - Keep a really big patent portfolio, so you can sue them right back.
>>
>> Ericsson is not small and uninteresting, but they _can_ exercise
>> caution in releasing
>> source code. Given this, it is actually no small matter that they
>> maintain Erlang/OTP
>> in the open. :)
>>
>> But hey, it's their decision. I'm not saying it's necessarily a bad
>> idea - just guessing
>> at some of the possible impediments.
>>
>> BR,
>> Ulf W
>>
>> On 27 Oct 2010, at 09:49, Edmond Begumisa wrote:
>>
>>> That puts a nice perspective on things... obvious now I was trying to
>>> use included apps for something they were not designed for. But your
>>> story really got me thinking...
>>>
>>> Sometimes I wish our friends at Ericsson would open-source one of
>>> their larger Erlang projects. Those of us who have NOT picked up
>>> Erlang from industrial environments have a problem of finding code we
>>> can reliably read for inspiration.
>>>
>>> I (and I think many), heard about Erlang from somewhere, went and
>>> read Joe's book (a superb introduction which covers the 'spirit' of
>>> Erlang), identified with the problems Erlang tries to solve, maybe
>>> read Cesarini and Thompson as a follow up (an excellent reference.)
>>> The problem is no matter how good these books are, there's only so
>>> much a book can teach you. A book certainly cannot advise you on how
>>> to write a complete application or how to avoid writing programs that
>>> become unwieldy as they grow. The question is what next?
>>>
>>> The obvious answer is to look at some open-source code. But when most
>>> open-source projects have learned Erlang the same way it becomes a
>>> case of the blind leading the blind. Problems that must have already
>>> been experienced and dealt with at Ericsson are repeated. A large
>>> production-ready open-source project written by battle-hardened
>>> experienced Erlang programmers would really fill in that void.
>>> Something big and sufficiently complex like a soft-switch or something.
>>>
>>> - Edmond -
>>>
>>>
>>> On Tue, 28 Sep 2010 17:07:16 +1000, Ulf Wiger
>>> <ulf.wiger@REDACTED> wrote:
>>>
>>>>
>>>> On 27 Sep 2010, at 20:05, Edmond Begumisa wrote:
>>>>
>>>>> Hi again Ulf,
>>>>>
>>>>> It's great to get 'the guy' on this subject online so I'm going to
>>>>> take full advantage and ask two more questions that have been
>>>>> dogging me...
>>>>>
>>>>> Firstly, is there an open-source project you know of that uses
>>>>> included-applications and/or start phases properly that I could
>>>>> take a peek at? Maybe in OTP source itself?
>>>>
>>>> Off the top of my head, I really can't think of any. :)
>>>>
>>>> The area where start phases really come in handy is when your
>>>> application needs
>>>> to support failover/takeover behaviour. This is also when the
>>>> StartType argument
>>>> is needed. One can implement takeover by writing a special start
>>>> phase that instructs
>>>> the processes to take over processing from the other side. In
>>>> general, it is best to do
>>>> this at a point where all processes have been started and are ready
>>>> to process
>>>> incoming requests.
>>>>
>>>> The initial reason for start phases was that the complex
>>>> call-handling applications
>>>> at Ericsson had some pretty horrendous dependencies to sort out
>>>> before they
>>>> could start accepting calls, and doing this work in the init
>>>> function of the processes
>>>> simply wasn't feasible. Also, when a process dies in the init
>>>> function, this is
>>>> interpreted as a start error, and the application start will fail,
>>>> whereas individual
>>>> processes have proper supervision while they are responding to
>>>> requests from
>>>> the start phase code (which runs in the application_starter process).
>>>>
>>>> Included applications were mainly introduced since the same
>>>> call-handling
>>>> applications needed to move as one during failover and takeover, and
>>>> starting
>>>> a dozen or so top applications made that much more difficult. It was
>>>> just too much
>>>> code and too many modules to integrate into one single application
>>>> without one
>>>> more structuring layer.
>>>>
>>>> Initially, I wrote some code that read .appSrc files in each
>>>> sub-application and
>>>> integrated them into one larger application, using a top-level
>>>> resource file - I
>>>> think it had the extension .appLm (as in load module - never mind;
>>>> it made sense
>>>> at Ericsson, and it was so long ago that I may be remembering
>>>> wrong). This was
>>>> later generalised by OTP into included_applications.
>>>>
>>>> The O&M applications also had a problem during takeover: The snmp code
>>>> assumed that the snmp agent was locally registered on the same node,
>>>> which
>>>> wasn't necessarily the case during the transition - either on the
>>>> node taking over
>>>> or on the node where it ran before. We then created a wrapper
>>>> application that
>>>> included all the O&M applications, and called the individual start
>>>> functions for
>>>> each included app.
>>>>
>>>> Later, we moved away from that, as we had to also support
>>>> applications that
>>>> were written according to a different timeline, and therefore
>>>> couldn't be integrated
>>>> the same way as our other apps. I came up with a solution for
>>>> starting and stopping
>>>> included apps and plugging in their start phase hooks in the right
>>>> places in the
>>>> startup flow, but for some reason people found it complicated... :)
>>>>
>>>> The better solution was to make use of the fact that the application
>>>> controller now
>>>> had a message passing interface for controlling the starting and
>>>> stopping of apps.
>>>> We were already using this in our cluster controller, so we could
>>>> extend it by
>>>> specifying distributed start dependencies and which applications
>>>> needed to do
>>>> takeover in parallel. This way, the cluster controller knew in which
>>>> order to move
>>>> applications during takeover, and in which order to terminate them,
>>>> once migrated.
>>>> Unfortunately, all this code is proprietary. It's on my long list of
>>>> things I'd like to do,
>>>> but that list just keeps growing, without much ever being removed
>>>> from it...
>>>>
>>>> A long time ago, I made a prototype (and sent to OTP) that
>>>> introduced start phase
>>>> dependencies. This would IMHO make it much easier to specify
>>>> dependencies
>>>> between applications. As an example, mnesia loads tables in the
>>>> background, so
>>>> when the application:start() function returns, one cannot assume
>>>> that tables are
>>>> loaded, and has to call mnesia:wait_for_tables() (which can time
>>>> out, and has some
>>>> corner cases where tables will never be loaded without intervention
>>>> - not that the
>>>> function itself will tell you when they occur). It might be better
>>>> if mnesia had a
>>>> load_tables start phase, which other applications could depend on.
>>>>
>>>> BR,
>>>> Ulf W
>>>>
>>>>>
>>>>> Secondly, I've always liked the idea of using included applications
>>>>> not necessarily for start phases but as a delayed/start-on-demand
>>>>> mechanism (taking advantage of the fact that included apps are
>>>>> automatically loaded but not started.) That is, manually calling
>>>>> application:start(foo) only if a particular feature of my app is
>>>>> used. But I have one query that made attempts for such use
>>>>> short-lived... the fact that an application can only be included by
>>>>> one other application. I think this limitation makes it harder to
>>>>> use included apps and start phases especially if you're using apps
>>>>> that are not in-house.
>>>>>
>>>>> For example, lets say CouchDB starts using mnesia (ok that's dumb
>>>>> but...) and decide to start it up using start phases (and therefore
>>>>> add it as an included application in couch.app) Then I have my
>>>>> FunkyApp that's been using mnesia too as included application. I
>>>>> then decide to use CouchDB for a new funky feature of FunkyApp. Now
>>>>> things break because mnesia is being used by both FunkyApp and
>>>>> CouchDB. To fix this, I not only have to modify my in-house app I
>>>>> have to modify the out-house CouchDB too.
>>>>>
>>>>> Is there an obvious fix to this I've been missing?
>>>>>
>>>>> - Edmond -
>>>>>
>>>>> On Tue, 28 Sep 2010 03:19:29 +1000, Ulf Wiger
>>>>> <ulf.wiger@REDACTED> wrote:
>>>>>
>>>>>
>>>>> On 27 Sep 2010, at 18:14, Edmond Begumisa wrote:
>>>>>
>>>>> Ulf,
>>>>>
>>>>> I've been doing such initialisation in the init function of a
>>>>> worker manager process. Using Daniel's example, I might have a
>>>>> gen_server child of the main supervisor called db_mgr and set up
>>>>> the mnesia schema in db_mgr:init
>>>>>
>>>>> Have I been doing the 'wrong' thing OTP-wise?
>>>>>
>>>>> Not necessarily, but my personal preference is to cleanly separate
>>>>> setup code
>>>>> from application startup. This is in part because I used to work on
>>>>> a very complex
>>>>> product, where the setup was decidedly non-trivial, and the startup
>>>>> process had
>>>>> to be optimised in several steps.
>>>>>
>>>>> Still, even there, I believe that the setup logic was bootstrapped
>>>>> into the startup
>>>>> phase, but the code was still kept cleanly separated. The only
>>>>> thing that was
>>>>> part of the startup was a simple check to see if the setup code had
>>>>> been run.
>>>>>
>>>>> BR,
>>>>> Ulf W
>>>>>
>>>>>
>>>>>
>>>>> - Edmond  -
>>>>>
>>>>> On Tue, 28 Sep 2010 00:31:47 +1000, Ulf Wiger
>>>>> <ulf.wiger@REDACTED> wrote:
>>>>>
>>>>> On 27/09/2010 16:15, Daniel Goertzen wrote:
>>>>> I've read the documentation on supervision and have seen a few
>>>>> tutorials,
>>>>> but they don't seem to move beyond the core concepts.  For example,
>>>>> what
>>>>> happens if you want to check and optionally setup an mnesia schema
>>>>> during
>>>>> startup...where should this code go?  In the supervisor init() or
>>>>> start_link() function?  Should I have my supervisor create a worker
>>>>> process
>>>>> whole sole job is to do this kind of setup and then dynamically add
>>>>> other
>>>>> workers (or supervisors) to the supervisor with start_child()?
>>>>>
>>>>> I strongly recommend doing that sort of thing in a separate procedure,
>>>>> rather than in the startup phase.
>>>>>
>>>>> If you want your application to be able to bootstrap itself, I would
>>>>> suggest that you either:
>>>>>
>>>>> - create a special application that runs before your other apps,
>>>>> and verifies that the installation is ok. To this end, it might be
>>>>> useful to know that you can pre-sort the .rel file. The systools lib
>>>>> will only change the sort order if needed to respect start
>>>>> dependencies.
>>>>> - Introduce start_phases, then do minimal work in the init function,
>>>>> and push the rest to functions that are called from start phase
>>>>> hooks. This also has the advantage that you know that your processes
>>>>> are all started and ready to respond during the init phase.
>>>>>
>>>>> Start phases are documented in
>>>>> http://www.erlang.org/doc/apps/kernel/application.html#Module:start_phase-3
>>>>>
>>>>>
>>>>> BR,
>>>>> Ulf W
>>>>>
>>>>>
>>>>>
>>>>> -- 
>>>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>>>>>
>>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd.
>>>>> http://erlang-solutions.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ________________________________________________________________
>>>>> erlang-questions (at) erlang.org mailing list.
>>>>> See http://www.erlang.org/faq.html
>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>>>>
>>>>>
>>>>>
>>>>> -- 
>>>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>>>>
>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd.
>>>> http://erlang-solutions.com
>>>>
>>>>
>>>>
>>>>
>>>> ________________________________________________________________
>>>> erlang-questions (at) erlang.org mailing list.
>>>> See http://www.erlang.org/faq.html
>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>>>
>>>
>>>
>>> -- 
>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>>
>> Ulf Wiger, CTO, Erlang Solutions, Ltd.
>> http://erlang-solutions.com
>>
>>
>>
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>
> 
>

More information about the erlang-questions mailing list