Port driver memory free

[Casper]

Hi Raimo,

Thank you so much for enlighten me on the subject. I was very unclear about
the binary mode and list mode and I thought even on the Binary mode I can
still use the already allocated memory in the **rbuf, if the total output is
less than 64 bytes.

Today I loaded my Erlang work and used "TOP" unix command to monitor the
Beam process and the CPU load. I noticed that Beam process memory size goes
up. Initially it was consuming about 36MB. But after about 4 hours later,
i.e. after passing more than 800,000 messages between Erlang and C Port, I
noticed that memory usage of Beam has gone up to 100MB.

As you instructed I don't use driver_free_binary. Could that be because I
don't free it? I checked my C code thoroughly but cannot find any other
memory leak area.

Please advice!

Thanks!
- Eranga





-----Original Message-----
From: owner-erlang-questions@REDACTED
[mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen
Sent: Tuesday, March 22, 2005 3:06 PM
To: erlang-questions@REDACTED
Subject: Re: Port driver memory free

Ok, to repeat once again. For port_control in binary mode, you must
_always_ return a binary. You can _not_ return the result in the
preallocated buffer *rbuf -it can only be used in non-binary
mode.

So, your loop that runs twice, once for size determination, and
once for encoding, should end in:
	if (p)
            break;
        *rbuf = driver_alloc_binary(*rindex);
	drv_binary = (ErlDrvBinary *) *rbuf;
	p = drv_binary->orig_bytes;
	*rindex = 0;
    }
Now p always gets set for the second iteration, and *rbuf always
gets set to a binary. That ought to work.


casper2000a@REDACTED writes:

> Raimo,
> 
> Actually I allocate a binary. Here's what I do,
> 
> ----------------------- C Port ----------------------------- 
> int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int
buflen) {
> 	ErlDrvBinary 	*drv_binary;
> 	char 			*p;
> 
> 	*rindex = 0;
> 	........
> 	........
> 	for (p = NULL;;) {
> 		if (ei_encode_version(p, rindex)
> 				|| ei_encode_tuple_header(p, rindex, 2)
> 				|| ei_encode_atom(p, rindex, "ok")
> 				|| ei_encode_binary(p, rindex, buffer,
buflen)) {
> 			DBG("ei_encode failed");
> 			return((int) ERL_DRV_ERROR_ERRNO);
> 		}
> 		if (p)
> 			break;
> 		//if (*rindex > rlen) {
> 			*rbuf = driver_alloc_binary(*rindex);
> 		//}
> 		drv_binary = (ErlDrvBinary *) *rbuf;
> 		p = drv_binary->orig_bytes;
> 		*rindex = 0;
> 	}
> 
> 	drv_binary->orig_size = *rindex;
> 	........
> ----------------------------------------------------------
> 
> I had to comment out the "(*rindex > rlen)" part since when it was there,
what I received in the 
> emulator side was garbage after 8th char position.
> 
> ------------------------- Emulator -----------------------
> case catch binary_to_term(erlang:port_control(Port, Command,

> 				term_to_binary(Args))) of 
> 	{'EXIT', Reason} ->
> 	.......
> ----------------------------------------------------------
> 
> On the emulator side binary_to_term BIF didn't give any error. But once
receive the term, the binary 
> part contain incorrect data after 8th char pos. 
> 
> Thanks!
> - Eranga
> 
> 
> 
> 
> -----Original Message-----
> From: owner-erlang-questions@REDACTED
> [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen
> Sent: Friday, March 18, 2005 2:31 PM
> To: erlang-questions@REDACTED
> Subject: Re: Port driver memory free
> 
> Yes, you do it wrong. The returned value _must_ be a binary, 
> (or NULL for R10B or later). The code that takes care of the value
> assumes it is a binary, so you have introduced something in your
> system that is assumed to be a binary, but is actually stack allocated
> plain memory. The emulator will probably crash much later because
> memory has been trashed. 
> 
> You will get partially correct behaviour because the receiving code 
> looks for the contents of the binary, and finds something there. 
> Unfortunately the header is not correct and it has not been allocated 
> with driver_alloc_binary(), but will later be freed with 
> driver_free_binary(), and both the contents and header will be
> overwritten with garbage as soon as the emulator C stack grows.
> 
> Do not do that - you will get hurt!
> 
> 
> 
> casper2000a@REDACTED (Casper) writes:
> 
> > Hi,
> > 
> > Today I noticed a funny thing with Port drivers \"control\" call.
> > 
> > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is
> (without
> > reinitializing), since the space I need to output the binary message is
> less
> > than 64 bytes.
> > 
> > When I decode the received binary message in the emulator side, up to 7
> > bytes were exactly as I encoded in the C Port side. After that 8
> position
> > onward I get some other value.
> > 
> > Then I created a new binary using driver_alloc_binary, even though the
> > output (encoded, total) is less than 64 bytes and tried, and viola, the
> > correct message came to emulator side.
> > 
> > If that a bug or something I do wrong?
> > 
> > Thanks!
> > - Eranga
> > 
> > 
> > 
> > -----Original Message-----
> > From: owner-erlang-questions@REDACTED
> > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen
> > Sent: Thursday, March 17, 2005 2:17 PM
> > To: erlang-questions@REDACTED
> > Subject: Re: Port driver memory free
> > 
> > The short answer is: do not free! 
> > 
> > To elaborate:
> > * In binary mode, you must return a binary:
> > ErlDrvBinary *binp = driver_alloc_binary(len);
> > /* Copy len bytes of data to to binp->orig_bytes */
> > *rbuf = (char *)binp;
> > return len;
> > The caller (the emulator) will increment the refcount and take care 
> > of deallocation when the binary is no longer used.
> > * In list mode, if the supplied result buffer is too small, you
> > should allocate a new result buffer using driver_alloc(size),
> > and the caller (the emulator) will free it it has fetched the
> > result from it. Otherwise just write the result in the supplied
> > result buffer. The emulator compares the returned buffer address
> > with the supplied buffer address to see if you return an allocated
> > buffer. So, if you set *rbuf, it _must_ be to something allocated 
> > with driver_alloc(size), because it _will_ be freed with 
> > driver_free(*rbuf). You can of course return an allocated buffer
> > for any size, also less than 64 bytes.
> > 
> > 
> > 
> > casper2000a@REDACTED (Casper) writes:
> > 
> > > Hi All,
> > > 
> > > There are 2 parameters, **rbuf and rlen in Port Driver entry
\"control.
> In
> > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in
> **rbuf.
> > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use
> that
> > > without creating a new ErlDrvBinary. My questions are,
> > > 
> > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf
> without
> > > allocating a new ErlDrvBinary, should I still call driver_free_binary
> or
> > > driver_free to free that **rbuf memory before \"control\" function
> returns?
> > > 
> > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new
> > > driver_binary and assign to **rbuf, what will happen to the previous
> 64
> > > buffer which was in **rbuf? Will the port or erlang garbage collecter
> free
> > > that? Or do I need to free it inside the \"control\" function by
calling
> > > driver_free_binary or driver_free?
> > > 
> > > 3. I (2) above, what will be the case if I use driver_realloc or
> > > driver_realloc_binary? Will the previous 64 byes get released?
> > > 
> > > 4. I (2) above, I still need to call driver_free_binary to free the
> newly
> > > created driver_binary inside \"control\" function, correct?
> > > 
> > > 5. If I call \"set_port_control_flags(dd->port, 0)\" to set the port
> output
> > to
> > > non-binary, do I need to free or clear the non-used space of **rbuf?
> > > 
> > > 6. In above cased which free function, driver_free_binary or
> driver_free
> > and
> > > which allocation function, driver_alloc, driver_alloc_binary,
> > > driver_realloc, driver_realloc_binary should I use?
> > > 
> > > Thanks in advance!
> > > - Eranga
> > > 
> > > 
> > 
> > -- 
> > 
> > / Raimo Niskanen, Erlang/OTP, Ericsson AB
> > 
> 
> -- 
> 
> / Raimo Niskanen, Erlang/OTP, Ericsson AB
> 
> --------------This mail sent through OmniBIS.com--------------
> 

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB