[erlang-questions] fallocate(FALLOC_FL_PUNCH_HOLE) / lseek(SEEK_HOLE / SEEK_DATA) anytime soon ?
Tue Nov 20 09:28:22 CET 2012
thanks for your answer
Le 19/11/2012 22:04, Richard O'Keefe a écrit :
> On 20/11/2012, at 6:21 AM, Bernard Fouché wrote:
>> Hi List,
>> Is there any chance that the API support these features in a reasonable time?
> Out of three operating systems immediately available to me,
> two (Solaris, Mac OS X) do not have fallocate, and one (three different
> versions of Linux that I am counting as one) does have fallocate() but
> insists that it has only an FALLOC_FL_KEEP_SIZE flag.
man 2 fallocate reports the same for me on Fedora 17 however
FALLOC_FL_PUNCH_HOLE is listed in /usr/include/linux/falloc.h . I guess
the man page is obsolete since kernel 2.6.38:
http://man7.org/tlpi/api_changes/index.html#Linux-2.6.38 . It seems that
Solaris has fcntl(F_FREESP) which provides the same feature. I can't
tell about Mac OS X. The feature is available in more and more fs (zfs,
xfs, etx4, btrfs and others).
With large files (2^64), in my use case (fixed size keys and fixed size
values), I should be able to avoid indexing a table since I can lseek to
(item_number * item_size) and free the corresponding disk block(s) when
I don't need the data anymore, instead of managing a list of free blocks
or instead of merging different files that handles deleted data by
marking it with a tombstone value. Data will be fragmented but with a
copy-on-write fs it is anyway and I guess it will be faster than to
process an index or merging files. BTW some of the mentioned fs also
have a defragment tool that can run on a live system.
> At its best, fallocate is Linux-specific, and posix_fallocate has
> no 'mode' argument, just fd, offset, and size.
> All of these operating systems support very large files, and
> also support files with holes where nothing was ever created in the
> first place.
> There is clearly room for Erlang access to operating-system-specific
> features, through operating-system-specific modules.
I'm still an Erlang novice and didn't look yet at the details of the
interface with the underlying OS. I'll try to find how an IoDevice is
matched to a fd...
More information about the erlang-questions