file: module and character special files

[Patrik Nyblom]

Hi there,

Richard A. O'Keefe wrote:

>Patrik Nyblom <pan@REDACTED> wrote:
>	I get your point, but obviously the suggested patch would not
>	solve any real problem.
>	
>Wrong.  The problem of not being able to use /dev/null for testing,
>especially for *output* testing, is a real one.  This patch does not
>provide EVERY POSSIBLE way of naming /dev/null, but it does provide
>ONE way of doing so.  That's enough for testing.
>
>	Erlang (BEAM) emulator version 5.4 [source] [threads:0]
>	
>	Eshell V5.4  (abort with ^G)
>	1> file:open("/dev/null",[read]).
>	{ok,<0.31.0>}
>	2> cd("/dev").                  
>	/dev
>	ok
>	3> file:open("null",[read]).
>	{error,eisdir}
>	4>
>
>That's not a problem.  On my own system, /dev/null is a symbolic link to
>/devices/pseudo/mm@REDACTED:null.  With my patch, that won't work.  I knew that.
>I don't *care*.  Anybody can create their own symbolic links to /dev/null,
>and they won't work either.  It would be nice if that worked, but I don't
>really care all that much if it doesn't.  What does matter is that there
>be *ONE* way that works.
>
>	This is of course easy to fix, but it indicates that the 
>	string-matching-magic-filename approach is no good.
>
>No, it simply means that it solves the problem _I_ wanted to solve,
>not the Aunt Sally you have set up in its place.
>
Right, but the problem _You_ wanted to solve is already solved, You did 
it, You have the source
and I trust you can compile it. It just isn't a solution general enough 
for my liking. Not when there's
a much better one around the corner.

The *real* problem, as I see it, is that arbitrary files cannot be 
opened by the file driver.

The problem with /dev/null is a bi-product of a larger problem, so to say.

>
>	Whoever wrote the original file driver draw the line at regular files, 
>	as that can be determined.
>	
>	Whoever that person was, I think he or she was right in doing this.
>	
>As I have pointed out, the answer is NO, he or she was clearly wrong.
>Remember, the stated reason for forbidding non-regular files is that
>they might be slow, BUT it has been the case for many years now that
>"regular" files might ALSO be slow.  Yes, it can be determined that
>a particular file name is classified by the operating system as a
>"regular file".  However, that does NOT mean that the file is on disc,
>and it does NOT mean that access to the file will be fast.
>
Granted, the assumption that a regular file is fast doesn't hold 
completely.
I think though, that it was a good choise to assume it does (at that time).

I failed to explain why I thought he/she was right.
To understand that reasoning, one must understand the environment in 
which Erlang
was used at the time (and still is used by most commercial customers).
Erlang, in telecom systems, runs on dedicated hardware. Usually with 
local file systems
and in rare cases with NFS solution where fairly high availability is 
implemented.
The networks are redundant and in case of network congestion or failure, 
the whole node fails and restarts.

In such an environment, it's a fairly safe assumption that regular file 
I/O won't block for unreasonably long
intervals. Thats the situation OTP is implemented for, at least when it 
comes to realtime properties.

For someone working in such an environment, the assumption that regular 
files are fast "almost" holds. It holds enough
to be sufficient. 

>
>The right thing to do is to allow ALL files (or possibly all files except
>directories, in which case the error message would be right) and just
>warn the programmer "it's up to you to ensure that the files you name
>are in fact fast files, because we can't easily tell".  Erlang has no
>reason to believe that any particular device not known to it is not fast,
>and no reason to believe that any particular regular file is not slow.
>
>	It is true that NFS can make reading from regular files block.
>	We cannot handle that and we cannot "see" if a file is on local or
>	remote filesystem from the driver.  The OS'es don't provide sufficient
>	interfaces to del with such things.
>	
>I must have been imagining the existence of the fstatvfs() system call.
>
>Having opened the file and got an fd,
>
>    struct statfvs b;
>    
>    if (fstatvfs(fd, &b) != 0)
>        check b.f_basetype[]
>    } else {
>        only documented error is some result too big to fit,
>        so maybe assume safe
>    }
>
>where values for .f_basetype[] include
>    tmpfs		The "temporary" file system
>    ufs			UNIX File System
>    nfs			NFS
>amongst others.  That will discriminate NFS files from local files,
>although, sadly, it won't discriminate /dev/tty from disc files.
>
I expressed myself inadequatly. I wrote "cannot see", which was of 
course wrong.
One *can* see, as you pointed out.

However, seeing doesn't help, as long as you don't have an interface 
sufficient
to do something about it.

Let's look at the options here. We see a file where I/O can block. It 
can either be a
device file, a FIFO or something where select/poll is possible, or it 
can be a file residing on a
remote host, an NFS file. What could we do?

1) We could refuse to open both FIFOs and remote files.
2) We could assume that regular files on NFS behave like local files, as 
they try to mimic
    local files in every other sense. FIFO's we refuse as they are 
obviously handled better by
    using select/poll.
3) We could ignore the problem and let the user take responsibility for 
his or her actions,
    meaning we would open any file without questioning.
4) We could implement select/poll for FIFO's (and character device 
files) and use
     AIO or another process to avoid blocking.
5) We could use threads and open anything without hesitation.

1) is obviously not a good idea. 2) is the "current" implementation, or 
at least the one we're discussing.
3) would not be all that bad, but unfortunately that would put us in a 
"support situation" we cannot handle.
4) has been tried but proved inadequate. AIO, as Scott pointed out in 
this thread, is not good enough.
5) Is great as long as you have kernel threads (Linux, Solaris, Window$, 
Darwin but not all BSD's).

Without threads, the only way we could have used the information that 
this is an NFS file, would have been
by using AIO on the file (suggestion 4 above). That proved to be much to 
hard to get portable and stable.
We would have done it anyway if no other solution would have been at 
hand, but it would have been inadequate
because of the problems with the "missing" AIO operations.

>
>Did I point out that there's a blocking problem with the code as it stands?
>Opening a file may itself block a process indefinitely.  Suppose a file
>name happens to refer to a named pipe, and you open it for reading.  Then
>the open() call will block until another process opens the thing for
>writing.  Then, *after* blocking Erlang for hours or even days, *then* it
>will notice "oh, it's not a regular file, eisdir".
>
Right you are. Thanx, I will personally change it to do stat before 
opening on Unix too.

>
>To fix that, it is necessary to check what kind of file it is
>BEFORE trying to open it, instead of after.  With Hierarchical Storage
>Management systems, the same kind of thing can happen:  try to open a
>file and get a big all-of-Erlang-blocking delay while it moves closer in
>the memory hierarchy.
>
>	We do not recommend people writing real-time systems relying on
>	NFS over unstable network connections either, but that's beside
>	the point.
>
>Erlang is useful for soft-real-time systems, but it is not ONLY useful
>for such systems.  DocBuild or whatever it's called that processes the
>Erlang documentation is written in Erlang, but it is hardly a real-time
>program.
>
You're correct, but it's not such a big problem if DocBuild, Wings3D or 
other
utility programs happen to block because of a slow NFS server. Realtime 
applications
where Erlang is used may affect miljons of people (or millions of $$$ 
:-), why hanging
emulators are not very popular among those customers. A DocBuild user 
might not even notice.

Performance, however *is* very popular, which is the reason for moving file
I/O into the emulator in the first place (before my time I have to admit).

The point is that the blocking I/O for regular files is (was) acceptable 
to those who
financed the OTP development, why it stayed around for long and is still 
there if you
don't enable I/O threads. It's not perfect, but it's fairly reliable and 
it's fast.


>
>	Tony Rogvall (together with others) has made a remarkable
>	*real, working, and portable* implementation
>	of a threaded file driver, submitted it to us, and that is the one 
>	nowdays present in the emulator.
>	
>Hooray!  Even an ordinary read on a local disc may take 10ms, which is
>not exactly fast these days.  When will this be released?
>
In minus three years :-) It's been around since shortly after the 
discussion four years ago.
erl +A256 gives an I/O thread pool of 256 threads. Even the FIFO bug 
won't block that emulator easilly...

However, to allow any file to be opened has not been implemented. I 
think one has to
give some means to dedicate threads for I/O on regular files, to avoid 
things like
FIFO's etc in wast ammounts clogging the thread pool completely. Apart 
from that
it's a matter of a few simple code-lines.

It's of course easy to patch the file driver to allow any file for you 
open source users,
so in conjunction with the +A switch, a working solution is already at 
hand for most of you.

>
>I note that the "opening a file to see what it is may delay arbitrarily
>long" problem is still there. It is necessary _first_ to check whether
>a file name refers to a FIFO before trying to open it.  The open() call
>needs to be kept from blocking all of Erlang.
>
>	Is there really anyone needing this?
>
>More precisely, is there anyone who _likes_ the idea of the fastest
>source/sink we've got being arbitrarily block in the name of getting
>a speed guarantee we do not in fact get?
>
>	A simple wrapper and some HTTP stuff.
>	
>Speaking of which, once again, does anyone know where I should be
>looking for the documentation of the http module?
>
>  
>
Best Regards,
/Patrik


This communication is confidential and intended solely for the addressee(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you believe this message has been sent to you in error, please notify the sender by replying to this transmission and delete the message without disclosing it. Thank you.

E-mail including attachments is susceptible to data corruption, interruption, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.


This communication is confidential and intended solely for the addressee(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you believe this message has been sent to you in error, please notify the sender by replying to this transmission and delete the message without disclosing it. Thank you.

E-mail including attachments is susceptible to data corruption, interruption, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.