<html><head>
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
</head><body bgcolor="#FFFFFF" text="#000000">Also, fwiw, here are links
to other people with the same issue over the years:<br>
<br>
<a class="moz-txt-link-freetext" href="http://erlang.org/pipermail/erlang-questions/2010-November/054330.html">http://erlang.org/pipermail/erlang-questions/2010-November/054330.html</a><br>
<a class="moz-txt-link-freetext" href="http://erlang.org/pipermail/erlang-questions/2010-October/053944.html">http://erlang.org/pipermail/erlang-questions/2010-October/053944.html</a><br>
<a class="moz-txt-link-freetext" href="http://erlang.org/pipermail/erlang-questions/2009-March/042123.html">http://erlang.org/pipermail/erlang-questions/2009-March/042123.html</a><br>
<a class="moz-txt-link-freetext" href="http://stackoverflow.com/questions/8792376/erlang-ports-interfacing-with-a-wc-like-program">http://stackoverflow.com/questions/8792376/erlang-ports-interfacing-with-a-wc-like-program</a><br>
<br>
And here are hacks that have been written to get around this limitation:<br>
<br>
<a class="moz-txt-link-freetext" href="https://github.com/mattsta/erlang-stdinout-pool#why-is-this-special">https://github.com/mattsta/erlang-stdinout-pool#why-is-this-special</a><br>
I assume this does as well: <a class="moz-txt-link-freetext" href="https://github.com/saleyn/erlexec">https://github.com/saleyn/erlexec</a><br>
<br>
Cheers!<br>
<br>
-Anthony<br>
<br>
<blockquote style="border: 0px none;"
cite="mid:E0D51E3E-3653-4B68-A506-F86D1A70397C@cs.otago.ac.nz"
type="cite">
<div style="margin:30px 25px 10px 25px;" class="__pbConvHr"><div
style="display:table;width:100%;border-top:1px solid
#EDEEF0;padding-top:5px"> <div
style="display:table-cell;vertical-align:middle;padding-right:6px;"><img
photoaddress="ok@cs.otago.ac.nz" photoname="Richard A. O'Keefe"
src="cid:part1.00010107.02030707@raynes.me"
name="compose-unknown-contact.jpg" height="25px" width="25px"></div> <div
style="display:table-cell;white-space:nowrap;vertical-align:middle;width:100%">
<a moz-do-not-send="true" href="mailto:ok@cs.otago.ac.nz"
style="color:#737F92
!important;padding-right:6px;font-weight:bold;text-decoration:none
!important;">Richard A. O'Keefe</a></div> <div
style="display:table-cell;white-space:nowrap;vertical-align:middle;">
<font color="#9FA2A5"><span style="padding-left:6px">July 29, 2013
10:55 PM</span></font></div></div></div>
<div style="color:#888888;margin-left:24px;margin-right:24px;"
__pbrmquotes="true" class="__pbConvBody"><pre wrap="">On 30/07/2013, at 2:13 AM, Per Hedeland wrote:
</pre><blockquote type="cite"><pre wrap="">FWIW, I definitely agree that this is a missing piece of functionality.
</pre></blockquote><pre wrap=""><!---->
Oh it's present in several languages. You could argue that it's missing.
But is it missing like a car radio in a car that lacks one,
or is it missing like an ejector seat in a car that lacks one?
</pre><blockquote type="cite"><blockquote type="cite"><pre wrap="">Note that "only send data to a command" and "only receive data from a
command" are the traditional ways for a UNIX program to communicate
with another over a pipe.
</pre></blockquote><pre wrap="">Well, it's basically the definition of the traditional pipeline concept
of the Unix shells, and pipes are obviously what you need to implement
it - but that doesn't preclude other uses of pipes. The zsh shell even
allows you to set up "bidirectional pipes" on the commandline.
</pre></blockquote><pre wrap=""><!---->
There are reasons why I don't use zsh. That's one of them.
</pre><blockquote type="cite"><blockquote type="cite"><pre wrap="">popen(<command>, "r") reads the output of
the command and popen(<command>, "w") writes to the input of the command.
</pre></blockquote><pre wrap="">popen() is effectively a convenience function to abstract away the
somewhat non-trivial application of pipe(), fork(), close(), and
execve() that is required to set things up correctly for two particular
and common usages of pipes in application code. (It is not used by
common shells to implement pipelines though.)
</pre></blockquote><pre wrap=""><!---->
Having implemented popen() for two other high level languages, I know that.
The fact that common shells do not use it is irrelevant.
</pre><blockquote type="cite"><blockquote type="cite"><pre wrap="">There isn't even any standard _term_ for talking about connecting to both
stdin and stdout of a command in UNIX, and that's because it's an
incredibly easy way to deadlock.
</pre></blockquote><pre wrap="">There is no need to have a term for it, since all you need is two pipes,
one for each direction
</pre></blockquote><pre wrap=""><!---->
Non-sequitur. If it's a thing you need to do often, it's a thing you
need to be able to talk about. When I've thought of doing it, I've
used the word David Bacon used in his SETL2 system, "pump". And then
I've used the phrase "looming disaster" and done something else.
</pre><blockquote type="cite"><pre wrap="">- and it's probably uncommon enough to not
warrant its own convenience function. And you can indeed easily deadlock
if you don't think about what you're doing, but I really doubt that this
is the reason for any absence of terminology or functions.
</pre></blockquote><pre wrap=""><!---->
It's not that you can easily deadlock,
it's that it's hard *NOT* to deadlock.
</pre><blockquote type="cite"><pre wrap=""> (the risk is of course reduced due to the fact that the
VM does non-blocking I/O).
</pre></blockquote><pre wrap=""><!---->
And *that* is the thing that saves Erlang. Of course, avoiding the
coding complexity of dealing with nonblocking I/O is one of the reasons
for using a multithreading language like Erlang.
</pre><blockquote type="cite"><blockquote type="cite"><pre wrap="">Just like it prevents C users from doing the same thing.
</pre></blockquote><pre wrap="">No, there is nothing that prevents C users from doing the same thing.
</pre></blockquote><pre wrap=""><!---->
You may have misunderstood me.
</pre><blockquote type="cite"><blockquote type="cite"><blockquote
type="cite"><blockquote type="cite"><pre wrap="">THE POPEN INTERFACE <<<< prevents C users doing this.
</pre></blockquote></blockquote></blockquote></blockquote><pre wrap=""><!---->Yes, all the other functions are there, and yes, if you desperately
want to program it, you can. But it is enough work that nobody
ever does this lightly.
For that matter, it's not beyond the abilities of, say, the glibc
authors, to extend their implementation of popen to support "r+"
or "w+" modes, if there were much demand for it. (Oddly enough,
Mac OS X 10.7.5 _does_ support "r+" mode, but the Linux I checked
does not. Weird.) I have _never_ been able to understand the
differences between Mac OS X and POSIX.
</pre></div>
<div style="margin:30px 25px 10px 25px;" class="__pbConvHr"><div
style="display:table;width:100%;border-top:1px solid
#EDEEF0;padding-top:5px"> <div
style="display:table-cell;vertical-align:middle;padding-right:6px;"><img
photoaddress="per@hedeland.org" photoname="Per Hedeland"
src="cid:part1.00010107.02030707@raynes.me"
name="compose-unknown-contact.jpg" height="25px" width="25px"></div> <div
style="display:table-cell;white-space:nowrap;vertical-align:middle;width:100%">
<a moz-do-not-send="true" href="mailto:per@hedeland.org"
style="color:#737F92
!important;padding-right:6px;font-weight:bold;text-decoration:none
!important;">Per Hedeland</a></div> <div
style="display:table-cell;white-space:nowrap;vertical-align:middle;">
<font color="#9FA2A5"><span style="padding-left:6px">July 29, 2013
7:13 AM</span></font></div></div></div>
<div style="color:#888888;margin-left:24px;margin-right:24px;"
__pbrmquotes="true" class="__pbConvBody"><pre wrap="">"Richard A. O'Keefe" <a class="moz-txt-link-rfc2396E" href="mailto:ok@cs.otago.ac.nz"><ok@cs.otago.ac.nz></a> wrote:
</pre><blockquote type="cite"><pre wrap="">On 29/07/2013, at 4:14 PM, Anthony Grimes wrote:
</pre><blockquote type="cite"><pre wrap="">and communicating with external processes in Erlang. They seem to have
at least one particular fatal flaw which prevents them from being very
useful to me, and that is that there is no way to close stdin (and send
EOF) and then also read from the process's stdout. For example, I cannot
use a port to start the 'cat' program which listens on stdin for data
and waits for EOF and then echos that data back to you. I can do the
first part, which is send it data on stdin, but the only way for me to
close it is to call port_close and close the entire process.
</pre></blockquote></blockquote><pre wrap=""><!---->
FWIW, I definitely agree that this is a missing piece of functionality.
I'm not sure how useful/important it is in the grand scheme of things,
but personally I could have used it on a couple of occasions. As you
mentioned, it's basically the equivalent of TCP shutdown() that is
needed, although shutdown() is perhaps a bit over-engineered - I've
never seen anyone use SHUT_RD or SHUT_RDWR...
Also the "opposite" functionality is already available for ports via the
'eof' option - i.e. you get informed that the other end has closed its
write side, but can still write data in the other direction.
</pre><blockquote type="cite"><pre wrap="">Note that "only send data to a command" and "only receive data from a
command" are the traditional ways for a UNIX program to communicate
with another over a pipe.
</pre></blockquote><pre wrap=""><!---->
Well, it's basically the definition of the traditional pipeline concept
of the Unix shells, and pipes are obviously what you need to implement
it - but that doesn't preclude other uses of pipes. The zsh shell even
allows you to set up "bidirectional pipes" on the commandline.
</pre><blockquote type="cite"><pre wrap="">popen(<command>, "r") reads the output of
the command and popen(<command>, "w") writes to the input of the command.
</pre></blockquote><pre wrap=""><!---->
popen() is effectively a convenience function to abstract away the
somewhat non-trivial application of pipe(), fork(), close(), and
execve() that is required to set things up correctly for two particular
and common usages of pipes in application code. (It is not used by
common shells to implement pipelines though.)
</pre><blockquote type="cite"><pre wrap="">There isn't even any standard _term_ for talking about connecting to both
stdin and stdout of a command in UNIX, and that's because it's an
incredibly easy way to deadlock.
</pre></blockquote><pre wrap=""><!---->
There is no need to have a term for it, since all you need is two pipes,
one for each direction - and it's probably uncommon enough to not
warrant its own convenience function. And you can indeed easily deadlock
if you don't think about what you're doing, but I really doubt that this
is the reason for any absence of terminology or functions.
But anyway, I don't see how any of this is relevant to the question at
hand. Opening a bi-directional connection between two processes by means
of a pair of pipes is exactly what erlang:open_port/2 *already does*
when you use 'spawn' (or 'spawn_executable' these days) to start an
external process. And it has been doing this since day one, and I can't
recall anyone complaining how this is hopelessly dangerous due to the
risk of deadlock (the risk is of course reduced due to the fact that the
VM does non-blocking I/O).
</pre><blockquote type="cite"><blockquote type="cite"><pre wrap="">This issue prevents Erlang users from doing any even slightly more than
trivial communication with external processes without having some kind
of middleman program that handles the creation of the actual process you
need to talk to and looks for a specific byte sequence to indicate 'EOF'.
</pre></blockquote><pre wrap="">Just like it prevents C users from doing the same thing.
</pre></blockquote><pre wrap=""><!---->
No, there is nothing that prevents C users from doing the same thing.
And even if they have to go to some effort to do it, it just means
having to write a bit more C - whereas the Erlang user can't write a bit
more Erlang to do just the small addition of "close the write side of
one of the pipes", even though the pipe pair is already there...
</pre><blockquote type="cite"><pre wrap="">Unix anonymous pipes are simply the wrong tool for the job in _any_
programming language.
The historic way to do "slightly more than trivial communication with
external processes" has been to set the external processes up as C nodes
or to use sockets.
</pre></blockquote><pre wrap=""><!---->
Using (TCP) sockets instead of pipes doesn't really change the "risk of
deadlock". In the case of the Erlang VM (i.e. open_port vs gen_tcp), it
may actually increase it, due to the existence of the passive and
{active, once} modes for sockets - another piece of functionality that
is "missing" from ports.
--Per Hedeland
</pre></div>
</blockquote>
</body></html>