[erlang-questions] Preallocated binary — save memory

[Erik Søe Sørensen]

>  May it be a good idea to add nif function that returns large
preallocated binary and then generate such data with <<Data/binary,
Chunk/binary>> so that no reallocation is done?
How about if you do that, without preallocating, just starting with <<>>?
That would at least lower the number of allocations to
O(log(number_of_chunks)).
And if that doesn't cut it, sure - do the same except hand-craft a BEAM
function which does the preallocation... it sounds like that is possible
without resorting to NIFs :-)


2013/4/10 Max Lapshin <max.lapshin@REDACTED>

> There is a problem in erlyvideo with generating mpeg-ts.
>
> I have list of video frames, they are binaries of 4K - 100K bytes size
> with some additional metadata. I need to split all these binaries in chunks
> of about 184 bytes and create lots of such chunks.
>
> When mpeg-ts transformation is written in C, large binary is allocated
> (size = sum(frame_size)*C) and all frames with additional data are written
> to this block chunk by chunk.
>
> Only one memory allocation is done and only one copy is done.
>
>
> When I write it in erlang, lots (thousands) of small binaries are
> allocated and iolist_to_binary makes another copy with another allocation.
>
> May it be a good idea to add nif function that returns large preallocated
> binary and then generate such data with <<Data/binary, Chunk/binary>> so
> that no reallocation is done?
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130410/e14c9bdb/attachment.htm>