[erlang-questions] supervise a python process

[Edmond Begumisa]

On Fri, 11 Feb 2011 22:11:36 +1100, Roberto Ostinelli  
<roberto@REDACTED> wrote:

> dear list,
>
> i need to integrate existing python code with a SOA erlang layer, which
> provides some distribution facilities.
>
> i'm thinking to use BERT to enable communication between the erlang layer
> and python. the main point to not use ports is that, afaik, that if a  
> python
> port crashes the whole erlang VM goes down, and the point of using erlang
> supervisors to provide fault-tolerancy gets lost.
>

Not exactly. There are two "types" of ports: a plain port and a port  
driver (AKA linked-in driver).

A *port* runs in it's own OS process - it will not crash the VM if it  
crashes itself.
A *port driver* is linked-in to the VM - if it crashes it will take the VM  
down with it.

You could take a second look at using a port for your purposes.

> therefore, i'm fishing for suggestions on how to enable erlang to somehow
> provide 'supervisor' functionalities for a python process.

 From Erlang, ports look [kinda] like linked processes. If a port crashes,  
you receive an {'EXIT',Port,Why} message and you can restart it. Likewise  
if the Erlang process that owns the port dies on the Erlang side, the  
python OS process will be terminated*. These fit nicely into enabling the  
"supervisor functionalities" you desire.

*If the port hangs however, that's a different story.

> the first thing
> that came to my mind is opening a TCP connection for the BERT  
> communication
> between an erlang and a python module, so that in case the connection  
> gets
> lost, the python module is instructed to exit while the erlang process
> restarts the python process (probably with a os:cmd/1 command or  
> similar).
>

There are two questions you should treat separately here...

1. How to best to do the inter-OS-process IPC
2. How best to encode and decode messages between OS processes once you've  
chosen the IPC technique.

For question 1)

You can use a port (don't immediately dismiss this), or you could indeed  
use TCP sockets. Either technique can be successfully worked into your  
supervision tree. Each technique has advantages over the other. Also, from  
my experience, both techniques may require you to implement some sort of  
heartbeat for long-running external code. The ultimate answer will be a  
mixture of application specificity and taste.

For question 2)

You could use the External Term Format, you could indeed use BERT, you  
could roll-your-own format. Personally, I prefer using UBF because it's  
compact, it comes with a contract checker, and most IPC I do is FSM-like.  
But if you've decided you like BERT, heck, use BERT.

> are there already attempts at this i might not be aware of, or any
> additional ideas you might recommend/consider?

Have a look at UBF. Norton's implementation is pretty flexible (it can  
even use JSON for message encoding)...

http://norton.github.com/ubf/ubf-user-guide.en.html

- Edmond -

>
> any input welcome.
>
> thank you,
>
> r.


-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/