Untimely garbage collection

[Scott Lystig Fritchie]

In preparing for the upcoming ACM PLI Erlang workshop, I'm suddenly
*very* interested in the issues Shawn raises.  I'll split my comments
& questions into two messages ... two short messages are harder to
ignore (intentionally or accidentally) than a single long message.
:-)

>>>>> "sp" == Shawn Pearce <spearce@REDACTED> writes:

sp> When the second gen_server gets the binaries, it sends them to the
sp> port using Port ! {self(), {command, List}}, where List is the
sp> List of ErlDrvBinary objects given to Erts by the bt848 driver.

The docs & erts code seem to imply that erlang:port_command/2 is the
preferred way of doing that.  {shrug}

Would I be correct to guess that your XVideo driver defines the
'outputv' method and that its 'outputv' handler accesses the pointers
inside of the ErlIOVec directly (to avoid unnecessary data copies)?

sp> Initial testing showed that allocating ErlDrvBinary objects for
sp> each video frame was far too costly in CPU time.  The allocator is
sp> just too slow.

Really?  You really be moving a *lot* of data through those drivers.
Or, if after allocating a ErlDrvBinary, you don't have enough time to
copy the frame into the new ErlDrvBinary without dropping some data?

Perhaps this strategy would be useful?  Have the bt848 driver allocate
a single (or a small number of) ErlDrvBinary large enough to hold
several frames worth of data.  The driver can choose the offset in a
ErlDrvBinary to deposit the next frame's data.  Hrm ... it isn't
obvious if this would lower your overhead or not.

sp> From the perspective of my application, it would be ok
sp> for my Erlang servers to notify the C drivers when they are done
sp> with the binary so it can rewrite it, regardless of the refc.

I had a brainstorm I had yesterday on this topic.  Consider this
example from the SWIG (http://www.swig.org/) documentation:

	# Copy a file
	def filecopy(source,target):
		f1 = fopen(source, "r")
		f2 = fopen(target, "w")
		buffer = malloc(8192)
		nbytes = fread(buffer,8192,1,f1)
		while (nbytes > 0):
			fwrite(buffer,8192,1,f2)
			nbytes = fread(buffer,8192,1,f1)
		free(buffer)

An Erlang driver cannot implement malloc and fread in this manner
because of its assumption of multiple assignment.

But, what if the local Erlang process knew that certain binaries were
multiple-assignment-capable?  Then it could safely work like filecopy
above *if* it were written carefully.  This might be viable if two
things were added:

	1. If the owner process of such a binary were to send it
	to another process, the ErlDrvBinary data would be _copied_ so
	that the multiple-assignment-ignorant receiver could
	blissfully assume single-assignment semantics?

	2. The driver implemented a copy method so that the owner
	process could make a single-assignment "snapshot" of the
	multiple-assignment binary for long-term keeping.

Is this a good idea?  {shrug}

-Scott