file: module and character special files

[Richard A. O'Keefe]

I asked why file: doesn't handle /dev/null correctly.

"Erik Stenman" <Erik.Stenman@REDACTED> and matthias@REDACTED
both pointed me to Patrik Nyblom's explanation from nearly four
years ago:

  http://www.erlang.org/ml-archive/erlang-questions/200003/msg00018.html

The argument goes like this:
(a) it is hard to get non-blocking I/O going portably across UNIXes.
(b) therefore the file: module relies on a driver which does blocking
    I/O, which only really works if the I/O *doesn't* block, means
    that only "fast" devices are supported.
(c) which means that only 'regular' files are supported.
(d) which is why /dev/null is rejected.
(e) which is why you get the "eisdir" error message.

Ad (e): there is no excuse for giving the wrong error message.
The error message should indicate plainly that only regular files
are supported; it certainly should NOT state something that is
patently untrue!  If nothing else is done, 4 years is quite long
enough for a badly misleading message to be KNOWN to lurk in the
system; time to fix it.

Ad (d): /dev/null is the fastest device there is.  Read from it,
and you're immediately told "end of file" without any peripheral
activity at all.  Write to it, and the bytes are immediately thrown
away without any peripheral activity at all.  Nyblom's argument
about tape drives blocking Erlang processes for long times is
NO ARGUMENT AT ALL against supporting /dev/null.  /dev/null is
so useful for testing that if the driver has to make a special case:
    if stat(2) says the file is a regular file
    or stat(2) says the file is a character special file
       and the name is /dev/null
    then open the file
    otherwise report "regular file needed"
then so be it.  I _think_ the requisite change is to
    erts/emulator/drivers/unix/unix_efile.c
in the function efile_openfile(), where the lines
    if (!ISREG(statbuf)) {
        close(fd);
        errno = EISDIR;
        return check_error(-1, errInfo);
    }
should be
    if (!ISREG(statbuf)
     && !(ISDEV(statbuf) && 0 == strcmp(name, "/dev/null"))
    ) {
        close(fd);
        errno = EISDIR;
        return check_error(-1, errInfo);
    }
This is also the place where something should be done about (e),
even if it does mean defining an ENOTREG error number that is not
one of the usual UNIX error numbers.

Ad (c): it has NOT been true for a long time that "operations won't
block" on regular files.  Not since RFS and NFS came into use.
The file system I see on my main machine is spread across three
different nodes; the machine I'm posting from brings in another.
I have known read() operations on what *looked* like "regular" files
take 30 seconds.  (On one memorable occasion, several hours.)

Ad (b): I don't really understand the present design (the internal
documentation of the file driver code is, shall we say, sparse), so
I can't comment.

Ad (a): while there may be UNIX systems whose implementors have
disgraced themselves by failing to make poll() or select() work
on files, in the light of the fact that "regular" file operations
can't really be trusted to be fast, it might be a good thing to have
a poll/select-based driver that could be used in UNIX systems where
those functions *do* work with files.  (This is the kind of thing that
./configure can figure out.)

Erik Stenman ended his reply thus:
	That post ends with:
	  "Maybe an alternative "any-type-of-file-driver" which uses
	   threads/separate processes would be nice though."
	
	Maybe we can get OTP to implement this if there are enough requests
	for a feature like that on the list, or maybe someone already has 
	such a driver somewhere that could be contributed to the community?
	
I was trying to "sell" the idea of using Erlang to a consultant.
What he _really_ wanted was a framework where he could open
URIs with at least the file, http, and ftp schemes either for input
or for output, like you can in Java.  Now, by using an outboard
program like cURL, this could be reduced to the problem of UNIX pipe
I/O, but that does mean that a driver that can handle both files
(with all the network delays that are implied by networked/remote/
shared file systems) and pipes (with all the delays implied by running
programs that may have to wait for other things to happen) would be of
use to Erlang.  It's a *much* bigger problem than just /dev/null, and
a much bigger opportunity.