Port driver memory free

[Casper]

Raimo,

Thanks a lot for your advice. I will check memory usage that way.

- Eranga


-----Original Message-----
From: owner-erlang-questions@REDACTED
[mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen
Sent: Tuesday, March 22, 2005 5:25 PM
To: erlang-questions@REDACTED
Subject: Re: Port driver memory free

It is not certain you have a leak. It may be just fragmented memory.
Try erlang:memory(), and/or also erlang:memory(binary) to see how
the emulator's internal view of allocated memory varies over the
time, and compare that to the crude figure from 'top'.

It is probably so that after your application has started it takes
some minutes / an hour for it to reach a quiescent state. And in
that state there is a lot of binaries allocated, being allocated
and being freed, so if erlang:memory(binary) asymtotically rises
to a mean value, fluctuating around it, there is no problem. If
on the other hand, erlang:memory(binary) rises linearily forever,
you have a memory leak of binaries. 

The value from 'top' can differ a lot from what erlang:memory()
reports, due to memory fragmentation.

You can also read "erl -man instrument" for an advanced feature
(I have never used) to analyze memory allocation in the emulator.



casper2000a@REDACTED (Casper) writes:

> Hi Raimo,
> 
> Thank you so much for enlighten me on the subject. I was very unclear
about
> the binary mode and list mode and I thought even on the Binary mode I can
> still use the already allocated memory in the **rbuf, if the total output
is
> less than 64 bytes.
> 
> Today I loaded my Erlang work and used "TOP" unix command to monitor the
> Beam process and the CPU load. I noticed that Beam process memory size
goes
> up. Initially it was consuming about 36MB. But after about 4 hours later,
> i.e. after passing more than 800,000 messages between Erlang and C Port, I
> noticed that memory usage of Beam has gone up to 100MB.
> 
> As you instructed I don't use driver_free_binary. Could that be because I
> don't free it? I checked my C code thoroughly but cannot find any other
> memory leak area.
> 
> Please advice!
> 
> Thanks!
> - Eranga
> 
> 
> 
> 
> 
> -----Original Message-----
> From: owner-erlang-questions@REDACTED
> [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen
> Sent: Tuesday, March 22, 2005 3:06 PM
> To: erlang-questions@REDACTED
> Subject: Re: Port driver memory free
> 
> Ok, to repeat once again. For port_control in binary mode, you must
> _always_ return a binary. You can _not_ return the result in the
> preallocated buffer *rbuf -it can only be used in non-binary
> mode.
> 
> So, your loop that runs twice, once for size determination, and
> once for encoding, should end in:
> 	if (p)
>             break;
>         *rbuf = driver_alloc_binary(*rindex);
> 	drv_binary = (ErlDrvBinary *) *rbuf;
> 	p = drv_binary->orig_bytes;
> 	*rindex = 0;
>     }
> Now p always gets set for the second iteration, and *rbuf always
> gets set to a binary. That ought to work.
> 
> 
> casper2000a@REDACTED writes:
> 
> > Raimo,
> > 
> > Actually I allocate a binary. Here's what I do,
> > 
> > ----------------------- C Port ----------------------------- 
> > int gen_reply(char **rbuf, int rlen, int *rindex, void *buffer, int
> buflen) {
> > 	ErlDrvBinary 	*drv_binary;
> > 	char 			*p;
> > 
> > 	*rindex = 0;
> > 	........
> > 	........
> > 	for (p = NULL;;) {
> > 		if (ei_encode_version(p, rindex)
> > 				|| ei_encode_tuple_header(p, rindex, 2)
> > 				|| ei_encode_atom(p, rindex, "ok")
> > 				|| ei_encode_binary(p, rindex, buffer,
> buflen)) {
> > 			DBG("ei_encode failed");
> > 			return((int) ERL_DRV_ERROR_ERRNO);
> > 		}
> > 		if (p)
> > 			break;
> > 		//if (*rindex > rlen) {
> > 			*rbuf = driver_alloc_binary(*rindex);
> > 		//}
> > 		drv_binary = (ErlDrvBinary *) *rbuf;
> > 		p = drv_binary->orig_bytes;
> > 		*rindex = 0;
> > 	}
> > 
> > 	drv_binary->orig_size = *rindex;
> > 	........
> > ----------------------------------------------------------
> > 
> > I had to comment out the "(*rindex > rlen)" part since when it was
there,
> what I received in the 
> > emulator side was garbage after 8th char position.
> > 
> > ------------------------- Emulator -----------------------
> > case catch binary_to_term(erlang:port_control(Port, Command,
> 
> > 				term_to_binary(Args))) of 
> > 	{'EXIT', Reason} ->
> > 	.......
> > ----------------------------------------------------------
> > 
> > On the emulator side binary_to_term BIF didn't give any error. But once
> receive the term, the binary 
> > part contain incorrect data after 8th char pos. 
> > 
> > Thanks!
> > - Eranga
> > 
> > 
> > 
> > 
> > -----Original Message-----
> > From: owner-erlang-questions@REDACTED
> > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen
> > Sent: Friday, March 18, 2005 2:31 PM
> > To: erlang-questions@REDACTED
> > Subject: Re: Port driver memory free
> > 
> > Yes, you do it wrong. The returned value _must_ be a binary, 
> > (or NULL for R10B or later). The code that takes care of the value
> > assumes it is a binary, so you have introduced something in your
> > system that is assumed to be a binary, but is actually stack allocated
> > plain memory. The emulator will probably crash much later because
> > memory has been trashed. 
> > 
> > You will get partially correct behaviour because the receiving code 
> > looks for the contents of the binary, and finds something there. 
> > Unfortunately the header is not correct and it has not been allocated 
> > with driver_alloc_binary(), but will later be freed with 
> > driver_free_binary(), and both the contents and header will be
> > overwritten with garbage as soon as the emulator C stack grows.
> > 
> > Do not do that - you will get hurt!
> > 
> > 
> > 
> > casper2000a@REDACTED (Casper) writes:
> > 
> > > Hi,
> > > 
> > > Today I noticed a funny thing with Port drivers \"control\" call.
> > > 
> > > I set the mode to PORT_CONTROL_FLAG_BINARY and use **rbuf as it is
> > (without
> > > reinitializing), since the space I need to output the binary message
is
> > less
> > > than 64 bytes.
> > > 
> > > When I decode the received binary message in the emulator side, up to
7
> > > bytes were exactly as I encoded in the C Port side. After that 8
> > position
> > > onward I get some other value.
> > > 
> > > Then I created a new binary using driver_alloc_binary, even though the
> > > output (encoded, total) is less than 64 bytes and tried, and viola,
the
> > > correct message came to emulator side.
> > > 
> > > If that a bug or something I do wrong?
> > > 
> > > Thanks!
> > > - Eranga
> > > 
> > > 
> > > 
> > > -----Original Message-----
> > > From: owner-erlang-questions@REDACTED
> > > [mailto:owner-erlang-questions@REDACTED] On Behalf Of Raimo Niskanen
> > > Sent: Thursday, March 17, 2005 2:17 PM
> > > To: erlang-questions@REDACTED
> > > Subject: Re: Port driver memory free
> > > 
> > > The short answer is: do not free! 
> > > 
> > > To elaborate:
> > > * In binary mode, you must return a binary:
> > > ErlDrvBinary *binp = driver_alloc_binary(len);
> > > /* Copy len bytes of data to to binp->orig_bytes */
> > > *rbuf = (char *)binp;
> > > return len;
> > > The caller (the emulator) will increment the refcount and take care 
> > > of deallocation when the binary is no longer used.
> > > * In list mode, if the supplied result buffer is too small, you
> > > should allocate a new result buffer using driver_alloc(size),
> > > and the caller (the emulator) will free it it has fetched the
> > > result from it. Otherwise just write the result in the supplied
> > > result buffer. The emulator compares the returned buffer address
> > > with the supplied buffer address to see if you return an allocated
> > > buffer. So, if you set *rbuf, it _must_ be to something allocated 
> > > with driver_alloc(size), because it _will_ be freed with 
> > > driver_free(*rbuf). You can of course return an allocated buffer
> > > for any size, also less than 64 bytes.
> > > 
> > > 
> > > 
> > > casper2000a@REDACTED (Casper) writes:
> > > 
> > > > Hi All,
> > > > 
> > > > There are 2 parameters, **rbuf and rlen in Port Driver entry
> \"control.
> > In
> > > > PORT_CONTROL_FLAG_BINARY mode, I should pass a driver_binary in
> > **rbuf.
> > > > Usually rlen is 64, which mean **rbuf has 64 bytes, if I need to use
> > that
> > > > without creating a new ErlDrvBinary. My questions are,
> > > > 
> > > > 1. If my final ErlDrvBinary size is < 64 bytes and I use **rbuf
> > without
> > > > allocating a new ErlDrvBinary, should I still call
driver_free_binary
> > or
> > > > driver_free to free that **rbuf memory before \"control\" function
> > returns?
> > > > 
> > > > 2. If my final ErlDrvBinary size is > 64 bytes and I allocate a new
> > > > driver_binary and assign to **rbuf, what will happen to the previous
> > 64
> > > > buffer which was in **rbuf? Will the port or erlang garbage
collecter
> > free
> > > > that? Or do I need to free it inside the \"control\" function by
> calling
> > > > driver_free_binary or driver_free?
> > > > 
> > > > 3. I (2) above, what will be the case if I use driver_realloc or
> > > > driver_realloc_binary? Will the previous 64 byes get released?
> > > > 
> > > > 4. I (2) above, I still need to call driver_free_binary to free the
> > newly
> > > > created driver_binary inside \"control\" function, correct?
> > > > 
> > > > 5. If I call \"set_port_control_flags(dd->port, 0)\" to set the port
> > output
> > > to
> > > > non-binary, do I need to free or clear the non-used space of **rbuf?
> > > > 
> > > > 6. In above cased which free function, driver_free_binary or
> > driver_free
> > > and
> > > > which allocation function, driver_alloc, driver_alloc_binary,
> > > > driver_realloc, driver_realloc_binary should I use?
> > > > 
> > > > Thanks in advance!
> > > > - Eranga
> > > > 
> > > > 
> > > 
> > > -- 
> > > 
> > > / Raimo Niskanen, Erlang/OTP, Ericsson AB
> > > 
> > 
> > -- 
> > 
> > / Raimo Niskanen, Erlang/OTP, Ericsson AB
> > 
> > --------------This mail sent through OmniBIS.com--------------
> > 
> 
> -- 
> 
> / Raimo Niskanen, Erlang/OTP, Ericsson AB
> 

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB