Erts doesn't support 64 bit files?

[Shawn Pearce]

So apparently OTP R8B-1 doesn't support 64 bit files...

	$ erl
	Erlang (BEAM) emulator version 5.1.1 [source] [hipe] [threads:0]

	Eshell V5.1.1  (abort with ^G)
	1> {ok, F} = file:open("test2.dat", [raw, binary, write]).
	{ok,{file_descriptor,prim_file,{#Port<0.26>,5}}}
	2> file:position(F, (1 bsl 31)-1).
	{ok,2147483647}
	3> file:write(F, "ab").
	File size limit exceeded
	$

Causes the entire system to crash.  This is on x86 Debian Potato,
using the 2.4.18 kernel.  It appears as though the crash
may actually be caused by the 32 bit write system call being
used by the efile driver, and not by erts.  Kind of hard to
guard against a crash caused by the C library.  :-(

This is interesting, because a user might be able to bring down
an Erlang node unknowningly.  A bug in a process may create an
invalid file position and bring the entire node down,
rather than just causing a single process to crash.

I'd settle for the node not crashing when a process writes
over the 2 GB barrier, but since I'm working on digital
video inside of Erlang I'd like true 64 bit file support.
I would really like to be able to keep the 4 or 8 GB video
streams in a single file whenever possible, especially
since the streams are stored on an XFS disk, which has a 14 TB
maximum file size.

So I'm politely submitting two feature requests: erts shouldn't
crash in the above case, and I'd really like to see larger file
support.  I doubt there is much that could be done to protect
erts from the 2 GB barrier crash, not without using a 64 bit
file interface.

In the mean time, I'm may write a new device driver that can
handle the 64 bit files on XFS, as well as to support XFS's Direct IO,
to bypass the kernel buffer cache.  Or just write a poor man's version
which splices the data across multiple 2GB files, and debug carefully.

--
Shawn.

Why do I like Perl?  Because ``in accordance with Unix tradition Perl
gives you enough rope to hang yourself with.''

Why do I dislike Java? Because ``the class ROPE that should contain the
method HANG to do the hanging doesn't exist because there is too much
'security' built into the base language.''