trouble with files bigger than 4GB

[Raimo Niskanen]

Not really a known bug, but I am not surprised. Large file support is 
fairly new, and not well (if at all :-) tested.

There is some way to compile a program for solaris so that fopen() means 
fopen64(), off_t becomes 64 bit, and so on. If this is done, the code 
should work. It is perhaps even so that for an Erlang/OTP system built 
for Solaris8 it works, while not for a system built for Solaris2.5.1.

Since the size of off_t is configured when building open source Erlang, 
on a system with 64 bits off_t - large file operations should work. Have 
a look in $ERL_TOP/erts/$PLATFORM/config.h for SIZEOF_OFF_T and see if 
it is 4 (bytes). $PLATFORM is the system name 
$ERL_TOP/erts/autoconf/config.guess reports when executed. $ERL_TOP is 
your open source build root.

Please try some more file operations on your large file. Try file:open, 
file:position, file:read, and see if it seems to work on FreeBSD. I am 
really curious.

I have browsed the source and think I see a bug in reading the size 
field for file:read_file_info - the high word of the size field is 
ignored, so if other file operations work, it is a small bug fix and can 
be done for R9C. If no other file operations work, it is a bigger 
problem and will have to wait. Please let me know.

/ Raimo Niskanen, Erlang/OTP, Ericsson AB



Daniel Solaz wrote:
> hi, anyone out there trying to deal with large files?
> 
> when given a file that is 4294967297 bytes (4GB + 1 byte) long, 
> file:read_file_info() incorrectly reports a file size of 1 byte
> 
> file:position(F, {eof, 0}) returns {ok, 1}
> 
> files bigger than 2GB but smaller than 4GB work fine
> 
> I'm using R9B-1 on FreeBSD 4.6 for intel
> 
> on Solaris 8 for intel, R9B-1 refuses to file:read_file_info() or 
> file:open() the file, returning {error, unknown}, and I actually prefer 
> this behavior
> 
> this difference might (or might not) be due to Solaris requiring 
> fopen64() for large file operations, whereas FreeBSD just uses fopen()
> 
> is this a known bug?
> 
> -Daniel