file: module and character special files

[Richard A. O'Keefe]

Patrik Nyblom <pan@REDACTED> wrote:
	I get your point, but obviously the suggested patch would not
	solve any real problem.
	
Wrong.  The problem of not being able to use /dev/null for testing,
especially for *output* testing, is a real one.  This patch does not
provide EVERY POSSIBLE way of naming /dev/null, but it does provide
ONE way of doing so.  That's enough for testing.

	Erlang (BEAM) emulator version 5.4 [source] [threads:0]
	
	Eshell V5.4  (abort with ^G)
	1> file:open("/dev/null",[read]).
	{ok,<0.31.0>}
	2> cd("/dev").                  
	/dev
	ok
	3> file:open("null",[read]).
	{error,eisdir}
	4>

That's not a problem.  On my own system, /dev/null is a symbolic link to
/devices/pseudo/mm@REDACTED:null.  With my patch, that won't work.  I knew that.
I don't *care*.  Anybody can create their own symbolic links to /dev/null,
and they won't work either.  It would be nice if that worked, but I don't
really care all that much if it doesn't.  What does matter is that there
be *ONE* way that works.

	This is of course easy to fix, but it indicates that the 
	string-matching-magic-filename approach is no good.

No, it simply means that it solves the problem _I_ wanted to solve,
not the Aunt Sally you have set up in its place.

	Whoever wrote the original file driver draw the line at regular files, 
	as that can be determined.
	
	Whoever that person was, I think he or she was right in doing this.
	
As I have pointed out, the answer is NO, he or she was clearly wrong.
Remember, the stated reason for forbidding non-regular files is that
they might be slow, BUT it has been the case for many years now that
"regular" files might ALSO be slow.  Yes, it can be determined that
a particular file name is classified by the operating system as a
"regular file".  However, that does NOT mean that the file is on disc,
and it does NOT mean that access to the file will be fast.

The right thing to do is to allow ALL files (or possibly all files except
directories, in which case the error message would be right) and just
warn the programmer "it's up to you to ensure that the files you name
are in fact fast files, because we can't easily tell".  Erlang has no
reason to believe that any particular device not known to it is not fast,
and no reason to believe that any particular regular file is not slow.

	It is true that NFS can make reading from regular files block.
	We cannot handle that and we cannot "see" if a file is on local or
	remote filesystem from the driver.  The OS'es don't provide sufficient
	interfaces to del with such things.
	
I must have been imagining the existence of the fstatvfs() system call.

Having opened the file and got an fd,

    struct statfvs b;
    
    if (fstatvfs(fd, &b) != 0)
        check b.f_basetype[]
    } else {
        only documented error is some result too big to fit,
        so maybe assume safe
    }

where values for .f_basetype[] include
    tmpfs		The "temporary" file system
    ufs			UNIX File System
    nfs			NFS
amongst others.  That will discriminate NFS files from local files,
although, sadly, it won't discriminate /dev/tty from disc files.

Did I point out that there's a blocking problem with the code as it stands?
Opening a file may itself block a process indefinitely.  Suppose a file
name happens to refer to a named pipe, and you open it for reading.  Then
the open() call will block until another process opens the thing for
writing.  Then, *after* blocking Erlang for hours or even days, *then* it
will notice "oh, it's not a regular file, eisdir".

To fix that, it is necessary to check what kind of file it is
BEFORE trying to open it, instead of after.  With Hierarchical Storage
Management systems, the same kind of thing can happen:  try to open a
file and get a big all-of-Erlang-blocking delay while it moves closer in
the memory hierarchy.

	We do not recommend people writing real-time systems relying on
	NFS over unstable network connections either, but that's beside
	the point.

Erlang is useful for soft-real-time systems, but it is not ONLY useful
for such systems.  DocBuild or whatever it's called that processes the
Erlang documentation is written in Erlang, but it is hardly a real-time
program.

	Tony Rogvall (together with others) has made a remarkable
	*real, working, and portable* implementation
	of a threaded file driver, submitted it to us, and that is the one 
	nowdays present in the emulator.
	
Hooray!  Even an ordinary read on a local disc may take 10ms, which is
not exactly fast these days.  When will this be released?

I note that the "opening a file to see what it is may delay arbitrarily
long" problem is still there. It is necessary _first_ to check whether
a file name refers to a FIFO before trying to open it.  The open() call
needs to be kept from blocking all of Erlang.

	Is there really anyone needing this?

More precisely, is there anyone who _likes_ the idea of the fastest
source/sink we've got being arbitrarily block in the name of getting
a speed guarantee we do not in fact get?

	A simple wrapper and some HTTP stuff.
	
Speaking of which, once again, does anyone know where I should be
looking for the documentation of the http module?