Untimely garbage collection
Mon Jul 15 09:35:25 CEST 2002
Your strategy should be working, I use more or less the same in
several drivers and it is the most efficient scheme I have figured
out so far.
1) the driver have two logical threads a) the vm itself and b) a
2) in the outputv thread a) make a copy of ErlIOVec,SysIOVec AND the
ErlDrvBinary vector in ErlIOVec. NOTE that it is simply the array of
pointers to the real ErlDrvBinaries. Store this in the drivers
3) For each element in the ErlDrvBinaries update refc.
4) signal is some way (i.e. use pipes) to thread b) to do the actual
processing and return from outputv. The vm may now do the gc as it
likes since the refc's are correct.
5) when thread b) is done with the chunk it signal over the pipe to
6) When ready_input is called in thread a) decrement the refc's for
all elements in the ErlDrvBinaries vector.
Deallocate the ErlDrvBinary vector, ErlIOVec and SysIOVec.
For a simple implementation use the the set_busy_port in step 2) and
6). The more ambitious approach is to allow multiple outstanding
> I know this isn't my problem. If I got the 30+ binaries, each
> over 128K apiece back in my shell, it'd choke.
> I'd dreged through most of the processes in my Erlang node, it
> doesn't look like they are stuck in anybody's message queue
> I had figured my strategy would work well, as it had been
> at one point about two weeks ago. I also checked most of
> the Erts source code, and in R8B-1 the binary should
> get collected and its refc updated immediately during
> a call to erlang:garbage_collect().
> I recently concluded about two days ago that I may not
> have all of the cases worked out on interaction between
> my background pthread and Erts. I think I am going to
> revisit them and see what might be missing.
> As an intersting side note, the refc's are messed up
> unless I use fprintf(stderr, "test!\r\n") in my
> driver. That causes the refc's to be correct. Cute.
> Clearly something is wrong.
> And I think Sean Hinde is right, what I am trying to
> do should just work, providing my thread communication is
> Sean Hinde <> scrawled:
> > > Suddenly this has stopped working. My C drivers are
> > > seizing when they run out of binaries, and all of the
> > > binaries have a refc of 3: one for the driver that
> > > "owns" the binary, and one for each gen_server process.
> > I had this problem once when developing my (now abandoned - mnesia
> > better :) ) SQL driver. The solution turned out to be sipmly that
> > testing from the shell which also retained the binaries in it's
> > history!
> > Otherwise binaries got GC'd very quickly and I could rely on the
> > problem.
> Why do I like Perl? Because ``in accordance with Unix tradition
> gives you enough rope to hang yourself with.''
> Why do I dislike Java? Because ``the class ROPE that should contain
> method HANG to do the hanging doesn't exist because there is too
> 'security' built into the base language.''
Synapse Systems AB
Phone: +46 709 686 685
More information about the erlang-questions