[erlang-questions] fallocate(FALLOC_FL_PUNCH_HOLE) / lseek(SEEK_HOLE / SEEK_DATA) anytime soon ?

Bernard Fouché bernard.fouche@REDACTED
Tue Nov 20 09:28:22 CET 2012

Hi Richard,

thanks for your answer

Le 19/11/2012 22:04, Richard O'Keefe a écrit :
> On 20/11/2012, at 6:21 AM, Bernard Fouché wrote:
>> Hi List,
>> Is there any chance that the API support these features in a reasonable time?
> Out of three operating systems immediately available to me,
> two (Solaris, Mac OS X) do not have fallocate, and one (three different
> versions of Linux that I am counting as one) does have fallocate() but
> insists that it has only an FALLOC_FL_KEEP_SIZE flag.
man 2 fallocate reports the same for me on Fedora 17 however 
FALLOC_FL_PUNCH_HOLE is listed in /usr/include/linux/falloc.h . I guess 
the man page is obsolete since kernel 2.6.38: 
http://man7.org/tlpi/api_changes/index.html#Linux-2.6.38 . It seems that 
Solaris has fcntl(F_FREESP) which provides the same feature. I can't 
tell about Mac OS X. The feature is available in more and more fs (zfs, 
xfs, etx4, btrfs and others).

With large files (2^64), in my use case (fixed size keys and fixed size 
values), I should be able to avoid indexing a table since I can lseek to 
(item_number * item_size) and free the corresponding disk block(s) when 
I don't need the data anymore, instead of managing a list of free blocks 
or instead of merging different files that handles deleted data by 
marking it with a tombstone value. Data will be fragmented but with a 
copy-on-write fs it is anyway and I guess it will be faster than to 
process an index or merging files. BTW some of the mentioned fs also 
have a defragment tool that can run on a live system.

> At its best, fallocate is Linux-specific, and posix_fallocate has
> no 'mode' argument, just fd, offset, and size.
> All of these operating systems support very large files, and
> also support files with holes where nothing was ever created in the
> first place.
> There is clearly room for Erlang access to operating-system-specific
> features, through operating-system-specific modules.
I'm still an Erlang novice and didn't look yet at the details of the 
interface with the underlying OS. I'll try to find how an IoDevice is 
matched to a fd...


More information about the erlang-questions mailing list