[erlang-questions] Issues with stdin on ports

[Anthony Grimes]

I'm not sure I follow. I think you may have misunderstood what I am
trying to accomplish. Here is a simple low-level example I just whipped
up in a Clojure repl using the standard built-in Java Process library.

user=> (def proc (.exec (Runtime/getRuntime) (into-array String ["cat"])))
#'user/proc
user=> (def stdin (.getOutputStream proc))
#'user/stdin
user=> (def stdout (.getInputStream proc))
#'user/stdout
user=> (.write stdin (.getBytes "Hi!"))
nil
user=> (.close stdin)
nil
user=> (let [arr (byte-array 3)] (.read stdout arr) (String. arr))
"Hi!"

Lots of unix programs work like this. We have cat in this example, but
grep, wc, and various others work like that as well, in that when you
run them they listen for data and wait for you to send EOF and then they
start producing output. It's a very common thing.

Every language I can think of lets you work with programs like this.
Ports are an abstraction for both reading and writing. The important
missing piece of functionality is being able to send EOF and tell the
program that you are done writing so that it can produce output. I'm not
following why this is a bad thing.

Perhaps this example and clarification will clear things up, and I
apologize in advance if I am missing something crucial!

-Anthony

> Richard A. O'Keefe <mailto:ok@REDACTED>
> July 29, 2013 12:15 AM
> On 29/07/2013, at 4:14 PM, Anthony Grimes wrote:
>
>> and communicating with external processes in Erlang. They seem to have
>> at least one particular fatal flaw which prevents them from being very
>> useful to me, and that is that there is no way to close stdin (and send
>> EOF) and then also read from the process's stdout. For example, I cannot
>> use a port to start the 'cat' program which listens on stdin for data
>> and waits for EOF and then echos that data back to you. I can do the
>> first part, which is send it data on stdin, but the only way for me to
>> close it is to call port_close and close the entire process.
>
> Note that "only send data to a command" and "only receive data from a
> command" are the traditional ways for a UNIX program to communicate
> with another over a pipe.  popen(<command>, "r") reads the output of
> the command and popen(<command>, "w") writes to the input of the command.
> There isn't even any standard _term_ for talking about connecting to both
> stdin and stdout of a command in UNIX, and that's because it's an
> incredibly easy way to deadlock.
>
>> This issue prevents Erlang users from doing any even slightly more than
>> trivial communication with external processes without having some kind
>> of middleman program that handles the creation of the actual process you
>> need to talk to and looks for a specific byte sequence to indicate 'EOF'.
>
> Just like it prevents C users from doing the same thing.
> Unless they fake something up using named pipes or UNIX-domain sockets.
> (Or message queues.  I do wish Mac OS X implemented rather more of POSIX...)
>
> Unix anonymous pipes are simply the wrong tool for the job in _any_
> programming language.
>
> The historic way to do "slightly more than trivial communication with
> external processes" has been to set the external processes up as C nodes
> or to use sockets.
>
>
>
>
> Anthony Grimes <mailto:i@REDACTED>
> July 28, 2013 9:14 PM
> Howdy folks.
>
> I unfortunately have not been able to use Erlang for most of what I've
> been doing lately because of a long standing issue with Erlang ports
> that I'd like to start a discussion about here.
>
> As far as I am aware, ports are generally the only option for creating
> and communicating with external processes in Erlang. They seem to have
> at least one particular fatal flaw which prevents them from being very
> useful to me, and that is that there is no way to close stdin (and send
> EOF) and then also read from the process's stdout. For example, I cannot
> use a port to start the 'cat' program which listens on stdin for data
> and waits for EOF and then echos that data back to you. I can do the
> first part, which is send it data on stdin, but the only way for me to
> close it is to call port_close and close the entire process.
>
> This issue prevents Erlang users from doing any even slightly more than
> trivial communication with external processes without having some kind
> of middleman program that handles the creation of the actual process you
> need to talk to and looks for a specific byte sequence to indicate 'EOF'.
>
> I could totally be wrong, but it seems like we need something other than
> just port_close. Something like
> http://www.erlang.org/doc/man/gen_tcp.html#shutdown-2
> <http://www.erlang.org/doc/man/gen_tcp.html#shutdown-2> which lets you say
> "Hey, I want to close the stdin of this process but still read from its
> stdout." or something similar. I could be totally off track on what a
> good solution would be.
>
> So I'm wondering if people are aware of this problem, and I'd like to
> make sure that people think it is an actual problem that should be
> fixed. I'm also curious what people think a good solution to the problem
> would be. I'm not sure I have the time/particular skill set to fix it
> given that the port code is some pretty obscure (to me) C code, but
> starting conversation seems like a good way to begin.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130729/35105963/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compose-unknown-contact.jpg
Type: image/jpeg
Size: 770 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130729/35105963/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: postbox-contact.jpg
Type: image/jpeg
Size: 1188 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130729/35105963/attachment-0001.jpg>