[erlang-questions] File size limits on file:write_file and file:write?

[Dave Challis]

On 04/05/11 13:33, Tony Rogvall wrote:
>
> On 4 maj 2011, at 13.49, Dave Challis wrote:
>
>> On 04/05/11 12:45, Ahmed Omar wrote:
>>> You wanted file:write_file/2 i guess
>>
>> Oops, that was a mistake in the original post.
>>
>> I've tried both:
>>
>> file:write_file("/tmp/foo", Data).
>>
>
> Nearly the same thing as below.
>
>> and
>>
>> {ok, Fp} = file:open("/tmp/foo", [write]).
>> file:write(Fp, Data).
>>
>> Both give {error,badarg} when Data > 2Gb, and write the file fine
>> otherwise.
>>
>
>
> Looks like the problem is in beam/utils/io_list_len.
> io_list_len returns an int, which is 32 bit on x64_64.
> The return type should probably be Sint, but the all calls to
> io_list_len must
> be updated accordingly.
>
> Workaround: write in smaller chunks.

Ah, thanks, this makes sense.  I'd just traced the error to:
erlang:iolist_to_binary([4, <<...3Gb binary...>>]).

which was failing, but wasn't sure where to look for iolist_to_binary's 
definition in the source (I'm assuming it uses io_list_len somewhere 
down the line).

It's something I'll watch for in future anyway, thanks again,

-- 
Dave Challis
dsc@REDACTED