[erlang-questions] exec-port crashing

[Michael Truog]

If you were using CloudI, each external service is supervised with a configuration setting for MaxR and MaxT in the same way it applies to a supervisor's one_for_one child process.  The external service OS process is forked from an Erlang port called cloudi_os_spawn.  To distribute the risk and latency of creating the external service processes, a cloudi_os_spawn OS process is created for each available OS CPU based on the configured (via the Erlang VM) scheduler count (a single cloudi_os_process is responsible for spawning all the configured number of processes as separate OS processes, each with a configured number of threads for parallel communication).  The only pipe communication with the external service processes is for capturing the stderr/stdout streams unbuffered for central logging.  The service communication uses a single socket per thread, due to a single instance of the CloudI API being created per thread.  That approach avoids the contention you have with a 
single cnode socket for a single cnode.  Using service request timeouts for communication and TCP, allows liveness checking of the external service without the net tick time that is required for cnodes.

So, while this may be slightly off-topic, the information may be useful for the concerns you have mentioned.  There is more information at http://cloudi.org/faq.html#4_Erlang

On 12/02/2015 09:30 AM, Serge Aleynikov wrote:
> Erlexec has been running in several production systems.  Yet, I can't claim that it's bug free as the application has many non-trivial features and handles various edge cases.  So if you find an issue, patches are welcome.
>
> It may become a bottleneck if you heavily rely on redirection of standard I/O streams of spawned processes, in which case erlexec does the I/O marshaling between file descriptors.  If that's the case, I suggest to redesign your application, so that it conveys I/O to Erlang by some other means, freeing erlexec from the role of being the "broker" and leaving it to do its main function - process supervision.
>
> On Tue, Dec 1, 2015 at 6:36 PM, Alberto Rosso <flagg.abe@REDACTED <mailto:flagg.abe@REDACTED>> wrote:
>
>     Hello guys,
>         our erlang application is running several c nodes and shell scripts, handled by erlexec. For some reason i'm going to debug deeper, the port exec-port crashes when decoding a string received from the erlang side.
>
>     This issue brought a couple of questions to light.
>     Isn't it bad to use exec-port as the unique gateway for all of my c nodes? I mean, a single crash of the exec-port frustrates the advantages of the supervision tree we've designed to make the application robust to hardware faults.
>     Another doubt is about the possibility for exec-port to become a performance bottleneck of the whole application, as the number of c nodes (and their throughput from/to the erlang app) is growing.
>
>     Could be starting more than one instance of the exec application, an (awful) solution?
>
>     Your thoughts are really appreciated.
>     -- 
>     Alberto Rosso
>     save a tree: please don't print this email unless strictly necessary
>
>     L'esperienza e la filosofia che non generano l'indulgenza e la carità sono due acquisti che non valgono quello che costano. (Alexandre Dumas figlio)
>
>
>     _______________________________________________
>     erlang-questions mailing list
>     erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>     http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20151202/c31dd381/attachment.htm>