[erlang-questions] fallocate(FALLOC_FL_PUNCH_HOLE) / lseek(SEEK_HOLE / SEEK_DATA) anytime soon ?

[Bernard Fouché]

Le 20/11/2012 23:52, Richard O'Keefe a écrit :
> On 20/11/2012, at 9:28 PM, Bernard Fouché wrote:
>>>> Is there any chance that the API support these features in a reasonable time?
>>> Out of three operating systems immediately available to me,
>>> two (Solaris, Mac OS X) do not have fallocate, and one (three different
>>> versions of Linux that I am counting as one) does have fallocate() but
>>> insists that it has only an FALLOC_FL_KEEP_SIZE flag.
>> man 2 fallocate reports the same for me on Fedora 17 however FALLOC_FL_PUNCH_HOLE is listed in /usr/include/linux/falloc.h . I guess the man page is obsolete since kernel 2.6.38: http://man7.org/tlpi/api_changes/index.html#Linux-2.6.38 . It seems that Solaris has fcntl(F_FREESP) which provides the same feature. I can't tell about Mac OS X. The feature is available in more and more fs (zfs, xfs, etx4, btrfs and others).
> I maintain a file comparing the APIs of
> 	Single Unix Specification version 3
> 	Single Unix Specification version 4
> 	Mac OS X (10.5 and 10.6; I don't have anything later at the moment)
> 	Linux
> 	Solaris (was Solaris 10, is now Solaris 11)
> 	HP-UX 11i (have manuals, not system)
> 	OpenBSD 4.7 (I really must upgrade this some time)
> 	Cygwin
> 	AIX (have manuals, not system; just begun on this one).
> because I'm trying to develop portable-to-POSIX-ish code.
lseek(SEEK_HOLE / SEEK_DATA) are about to be POSIX compliant 
(http://austingroupbugs.net/view.php?id=415) whenever POSIX 1003.1 2008 
is updated (this also brings an official definition of what is a sparse 
file). Adding SEEK_HOLE and SEEK_DATA in unix_efile.c seems easy, 
EFILE_SEEK_HOLE/DATA can return EINVAL if SEEK_HOLE/DATA are undefined 
for the current platform, or pass them down to lseek(2) if supported.

Unfortunately I wasn't able to find anything about hole punching in the 
Austin group discussions. So there are upcoming tools to know about 
holes but no planned tools to punch holes in the middle of a file, too 
bad...

> It's a huge amount of labour and really needs to be automated in some way.
>
> Out of that list, F_FREESP and F_FREESP64 are only available in Solaris.
> The Solaris documentation says
>
> 	 Free storage space associated  with  a  section  of  the
>           ordinary  file  fildes.  The  section  is specified by a
>           variable of data type struct flock pointed  to  by  arg.
>           The  data  type struct flock is defined in the <fcntl.h>
>           header (see fcntl.h(3HEAD)) and is described below. Note
>           that  all  file  systems  might not support all possible
>           variations of F_FREESP arguments.  In  particular,  many
>           file  systems allow space to be freed only at the end of
>           a file.
>
> It is not obvious to me why Linux didn't copy the Solaris interface.

And by changing the definition of fallocate() they also broke POSIX 
compliance.

> Freeing space at the end of a file can of course be done with
> {,f}truncate().
>
> Of course more recent versions of various operating systems may have
> all sorts of goodies.  I'm trying to avoid the bleeding edge.
>
I'll experiment and probably solve my problem by deporting hole 
management in an external C program, even if I would have preferred to 
keep the whole file management in the same piece of code.

   Bernard