[erlang-bugs] Bug in file I/O under Windows.

[Richard Kelsall]

Richard Carlsson wrote:
> John Hughes wrote:
>> On my Vista machine, it took about three quarters of a million iterations to
>> provoke a test failure.
> 
> The problem might be concurrent access by some other interfering program.
> I believe (someone correct me if I'm wrong) that under Windows, the default
> way of opening files for reading is exclusive (yes, even for read-only).
> This way, programs such as virus checkers and other things that traverse
> your file system for one reason or another (notably TortoiseSVN), can
> occasionally happen to prevent you from performing an operation even on
> a file that you just created and nobody is supposed to know about yet.
> 
> I'm not 100% sure that this is really what is happening, but it's our best
> hypothesis so far. We often see spurious failures when unzipping archives
> with loads of files, or sometimes when checking out big source trees from
> the repository. Only in Windows(TM). Oh, joy!
> 
>     /Richard
> 
> PS. What seems to be happening with unzip is this: unzip creates the file
> and populates it from the archive data, then closes it again. Then, as a
> separate operation it wants to set the timestamp to whatever it was in the
> archive. But, meanwhile, some greedy program is already checking out the new
> file, and we get error reports stating that the date could not be set. Gah!

This may not be relevant, but I'll add a bit of information about
a file problem I had under Windows in a Delphi program.

My problem was a virus checker and I got the impression that John
had ruled out virus checkers, but thinking about it, there might be
some sort of indexing or backup software running that performs the
same kind of file operations as a virus checker.

I have roughly edited a couple of old emails about my problem and
pasted them below, in case it helps. This was a Delphi program
running on Windows.

Richard.


When the program is installed it does lots of writing to files. A
virus checker may attempt to read these files immediately after they
are changed to look for viruses. The program may try another write
to the files while they are being read by the virus checker and fail
with the 'file in use' problem. (We found it could be solved by
switching off the virus checker. And it never occurred on my machine
without a virus checker.)

The relevant error messages were :
"The process cannot access the file because it is being used by
another process"
"File could not be opened. File may be in use by another process"
(I can't immediately tell you whether these are Delphi or Windows
messages.)

The cause of my 'File in use' problem is that a virus checker can
attempt to access the database tables while the program is running.
(It took an enormous effort to prove this.) The program requires
exclusive access to these files and can crash with the 'File in use'
problem if these files are accessed simultaneously by a virus checker
or similar application. It should be possible to reconfigure the virus
checker to ignore the files used by the program, or just run the virus
checker periodically rather than continuously.

This happens primarily when the program is installed because all
the records are loaded into the database at this point. This means
a huge number of database updates are performed in quick succession.
Nobody has yet reported the problem at other times but I don't see
why it shouldn't happen during normal use of the program. However,
during normal use database updates are brief and infrequent so even
if the problem occurs at the same rate per database update it will
almost never happen.

My technical guess at what is happening is that a file write operation
- in this case a database table change - is triggering the virus checker
to scan the newly updated file for viruses. During this scan the program
attempts to perform another write and falls in a heap because the virus
checker is using the file. It might be possible to modify my database
code to 'try again later' if the file is in use.