[erlang-questions] Preallocated binary — save memory

[Max Lapshin]

There is a problem in erlyvideo with generating mpeg-ts.

I have list of video frames, they are binaries of 4K - 100K bytes size with
some additional metadata. I need to split all these binaries in chunks of
about 184 bytes and create lots of such chunks.

When mpeg-ts transformation is written in C, large binary is allocated
(size = sum(frame_size)*C) and all frames with additional data are written
to this block chunk by chunk.

Only one memory allocation is done and only one copy is done.


When I write it in erlang, lots (thousands) of small binaries are allocated
and iolist_to_binary makes another copy with another allocation.

May it be a good idea to add nif function that returns large preallocated
binary and then generate such data with <<Data/binary, Chunk/binary>> so
that no reallocation is done?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130410/536e255f/attachment.htm>