Aggressive GC? Was Re: Untimely garbage collection

[Shawn Pearce]

My understanding of this (and the way I'm using it in my drivers)
is this:

When port_command/2 returns, the driver's outputv method
(or just output) has been called and returned already.  This
can be seen also when just using Port ! {self, {command, List}},
the driver's outputv function is called while the message is being
sent.  The send operation doesn't proceed to the next operation
in your Erlang code until the driver has completed its outputv
work.

Since the driver wants to hold the binary for some period of
time, it must increment the refc of any ErlDrvBinary which
it wants to keep, before it returns from the outputv
function.  This sets the refc to be one higher than
the number of Erlang processes still using the binary.

When the Erlang GC kicks in at your vunerability window,
it decrements the refc of the binary, but discovers that
the refc is still > 0, so it leaves the binary alone.

The driver calls driver_free_binary at some point in the
future, which decrements refc and determines that refc == 0,
so it deallocates the binary.

Therefore, the situation you are describing cannot occur.

Of course, since the driver is a pthread, you do have to
be very careful about the interaction with Erts.  If you are
using the async driver interface provided by Erts, you cannot
make emulator calls like driver_free_binary or driver_output_term
from within the background pthread.  They can only be made from
the main Erts thread that your erl_drv_entry functions are invoked
on.

In my bt848 driver case, I spawn my own pthread when the port
is setup.  This pthread uses a mutex and a pipe to communicate
with the Erts thread, and all Erts interactions occur on the
only when Erts calls my driver's erl_drv_entry functions.
In the outputv method of my driver, I increment refc, store them in
a memory space shared with the pthread, and signal the pthread to
wake up.  When the pthread is done with the binary, it
signals Erts over the pipe, and my ready_input driver method
decrements the refc (and deallocates the binary if
necessary).  Synchronization between the Erts thread and my
background pthread is done through a pthread mutex, which
is locked and unlocked in the outputv/read_input functions.

outputv:
	ev->binv[1]->refc++;	// I want to keep this binary!
	pthread_mutex_lock(&ref->lock);		// Sync with my pthread
	ref->the_binary = ev->binv[1];
	pthread_cond_signal(&ref->cond);	// Wakup pthread
	pthread_mutex_unlock(&ref->lock);	// Unsync 

ready_input:
	read(ref->pipe_in, &_junk, 1);
	pthread_mutex_lock(&ref->lock);
	driver_free_binary(ref->the_binary);
	ref->the_binary = NULL;
	pthread_mutex_unlock(&ref->lock);

bgthread_worker:
	pthread_mutex_lock(&ref->lock);
	for(;;)
	{
		while(!ref->the_binary)
			pthread_cond_wait(&ref->cond, &ref->lock);
		// Work with the_binary
		...
		// Signal Erts we are done.
		write(ref->pipe_out, &ref, 1);
	}
	pthread_mutex_unlock(&ref->lock);

The actual code is a little bit more complex, but this is the very
simple version of it.  Erts basically does the right thing, the
question is, does your driver?  ;-)

Plus, I don't think the GC would necessarily occur at a function
call.  I think you'd need to trip it by allocating memory or
calling erlang:garbage_collect() explicitly.

Scott Lystig Fritchie <fritchie@REDACTED> scrawled:
> While hacking drivers, I've come across a GC question, similar to
> Shawn's, that I cannot answer.  
> 
> Consider this code snippet:
> 
> 	iolist2binary(B) when binary(B) ->
> 	    {B, size(B)};
> 	iolist2binary(L) when List(L) ->
> 	    B = list_to_binary(L),
> 	    {B, size(B)}.
> 	
> 	foofoo(Port, IOList) when port(Port) ->
> 	    {IOListBinary, IOListBinaryLen} = iolist2binary(IOList),
> 	    C = [ <<?S1_FOOFOO, IOListBinaryLen:32/integer>>, IOListBinary],
> 	    erlang:port_command(Port, C),
> 	    %% Vulnerability window begins here?
> 	    get_port_reply(Port).
> 	
> 	get_port_reply(Port) ->
> 	    receive
> 	        {Port, ok} -> ok;
> 	        [...]
> 
> Assume:
> 
> 	1. The driver caches the pointer to IOListBinary's data
> 	buffer.  This permits the driver to avoid making a copy of
> 	that data buffer: it can access the binary's data buffer
> 	directly.
> 
> 	2. The driver is implemented asynchonously, using a separate
> 	Pthread.  Thus the VM can do other work while the driver's
> 	Pthread takes an arbitrary amount of time to do whatever it
> 	does.
> 
> My question: Is it possible for the VM's GC to decrement
> IOListBinary's refcount in the time between execution of the
> port_command() and the receive?  If the VM's GC were extremely
> aggressive(*), then if a GC happened at "Vulnerability window begins
> here?", then:
> 
> 	a. IOListBinary's refcount could go to zero: if the original
> 	IOList were a deep byte list, then foofoo() is the only thing
> 	that knows about IOListBinary, so its refcount would drop to
> 	zero.
> 
> 	b. Disaster!  The driver's cached pointer to IOListBinary's
> 	data buffer is invalid.  The driver is executing independently
> 	of the VM's Pthread, so there's no guarantee that the driver
> 	will use the pointer before the pointer becomes invalid.
> 
> -Scott
> 
> (*) If the GC system can figure out that a binding has not yet gone
> out of scope yet *but* that binding will never be used again, then
> ... I'm in trouble.
--
Shawn.

Why do I like Perl?  Because ``in accordance with Unix tradition Perl
gives you enough rope to hang yourself with.''

Why do I dislike Java? Because ``the class ROPE that should contain the
method HANG to do the hanging doesn't exist because there is too much
'security' built into the base language.''