[erlang-questions] Re: Possible leak in inets httpc.
Anthony Molinaro
anthonym@REDACTED
Fri Jul 9 18:26:48 CEST 2010
Yeah, I managed to get the entire set of patches, basically used
git diff 201ccdd08bd2a591fe1e348a9a1df3d94a3606e4 141ff3c087ca856d21d0 lib/inets
Which seemed to get them all. However, I still see memory leaked
Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0]
[kernel-poll:false]
Eshell V5.7.5 (abort with ^G)
1> inets:start().
ok
2> ets:i().
...
16402 httpc_manager__session_cookie_db bag 0 316 httpc_manager
httpc_manager__handler_db httpc_manager__handler_db set 0 316 httpc_manager
httpc_manager__session_db httpc_manager__session_db set 0 316 httpc_manager
...
3> http:request (post, { "http://www.example.com", [], "application/octet-stream", <<"hello">>}, [{connect_timeout, 100},{timeout, 10}], [{body_format, binary} ]).
{error,timeout}
4> ets:i().
...
16402 httpc_manager__session_cookie_db bag 0 316 httpc_manager
httpc_manager__handler_db httpc_manager__handler_db set 1 333 httpc_manager
httpc_manager__session_db httpc_manager__session_db set 0 316 httpc_manager
...
5> ets:i(httpc_manager__handler_db).
<1 > {handler_info,#Ref<0.0.0.142>,undefined,<0.60.0>,<0.33.0>,operational}
6> m(inets).
Module inets compiled: Date: July 8 2010, Time: 06.45
Compiler options: [{d,'SERVER_SOFTWARE',"inets/5.3.3"},
{cwd,"/home/molinaro/projects/work/vendor/erlang/R13B04/rpmbuild/BUILD/otp_src_R13B04/lib/inets/src/inets_app"},
{outdir,"/home/molinaro/projects/work/vendor/erlang/R13B04/rpmbuild/BUILD/otp_src_R13B04/lib/inets/src/inets_app/../../ebin"},
{attribute,insert,app_vsn,"inets-5.3.3"},
{parse_transform,sys_pre_attributes},
debug_info]
Object file: /usr/lib64/erlang/lib/inets-5.3.3/ebin/inets.beam
...
So it seems if you connect but then timeout you leak some memory even with
5.3.3. I've not tried 5.4 which is in dev but the only changes there seem
to be related to ssl...
Can anyone else confirm this is still broken?
-Anthony
On Fri, Jul 09, 2010 at 12:12:13PM +0200, Micael Karlberg wrote:
> This problem was fixed in inets-5.3.2. The fix was reported by Lev Walkin:
>
> http://www.erlang.org/cgi-bin/ezmlm-cgi/2/1766
>
> Unfortunately, github was not updated in one commit, but in several, here
> are some of them:
>
> http://github.com/erlang/otp/commit/4b87b22ce97abf2759eb551222a862e17c5f4dcb
> http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49
>
> So you have to do a bit of cut-and-paste. The last version of inets in
> R13B was 5.3.3,
> but I don't know if that made it into github.
>
> Regards,
> /BMK
>
> On Fri, Jul 9, 2010 at 8:12 AM, Andrey Popp <8mayday@REDACTED> wrote:
> > We have updated to R14 (thanks to deb packages in debian squeeze
> > repository) and all seems to be ok for now.
> >
> > On Fri, Jul 9, 2010 at 1:34 AM, Anthony Molinaro
> > <anthonym@REDACTED> wrote:
> >> Well, looks like the answer is yes and no. I was able to get a patch
> >> which applied cleanly, however, the problem does not appear to be fixed.
> >> The ets table still fills up with every request that times out
> >> (I unfortunately don't have a standalone test, but I've tested
> >> repeatedly and it always seem to get an extra entry for every request
> >> that times out).
> >>
> >> This is sort of a show stopper for me upgrading, so I'll be trying to
> >> figure out a fix, but if anyone more familiar with the code wants to
> >> help me out, it would be appreciated.
> >>
> >> Thanks,
> >>
> >> -Anthony
> >>
> >> On Thu, Jul 08, 2010 at 01:13:41PM -0700, Anthony Molinaro wrote:
> >>> Hi, anyone know if its safe to backport this fix to R13B04? I'm in the process
> >>> of deploying R13B04 as an upgrade from R12B05 and since I make heavy use of
> >>> httpc this will bite me. Ideally, I can just grab the appropriate files from
> >>> git and plop them into my R13B04 when I build an rpm (of course I need to
> >>> remember enough git to do that :) ), so wondering if it's possible or if
> >>> there are deps outside of the inets directory which matter?
> >>>
> >>> Thanks,
> >>>
> >>> -Anthony
> >>>
> >>> On Thu, Jul 08, 2010 at 02:12:07PM +0400, Andrey Popp wrote:
> >>> > It seems it has been fixed with [1] in inets-5.3.2. Sorry.
> >>> >
> >>> > [1]: http://github.com/erlang/otp/commit/91c89d54d45989a85367f10d5902b9b508754a49
> >>> >
> >>> > On Thu, Jul 8, 2010 at 1:57 PM, Andrey Popp <8mayday@REDACTED> wrote:
> >>> > > Hello,
> >>> > >
> >>> > > I have application that makes huge number of HTTP requests with httpc
> >>> > > and I have spotted that `httpc_manager__handler_db` ETS table growing
> >>> > > in size and doesn't clean up. That is the example records from this
> >>> > > table (ets:i(httpc_manager__handler_db)):
> >>> > >
> >>> > > <1 > {handler_info,#Ref<0.0.0.18800>,undefined,<0.177.0>,undefined,operational}
> >>> > > <2 > {handler_info,#Ref<0.0.0.35242>,undefined,<0.3075.0>,undefined,operational}
> >>> > > <3 > {handler_info,#Ref<0.0.0.61713>,undefined,<0.5755.0>,undefined,operational}
> >>> > > <4 > {handler_info,#Ref<0.0.0.68114>,undefined,<0.6943.0>,undefined,operational}
> >>> > > <5 > {handler_info,#Ref<0.0.0.85303>,undefined,<0.8305.0>,undefined,operational}
> >>> > >
> >>> > > and processes with handler pids are no longer alive. Also, note that I
> >>> > > have no timeout setting in my http:request HTTPOptions, but I think
> >>> > > inets should not leak even in that case.
> >>> > >
> >>> > > Thanks.
> >>> > >
> >>> > > --
> >>> > > Andrey Popp
> >>> > >
> >>> > > phone: +7 911 740 24 91
> >>> > > e-mail: 8mayday@REDACTED
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Andrey Popp
> >>> >
> >>> > phone: +7 911 740 24 91
> >>> > e-mail: 8mayday@REDACTED
> >>> >
> >>> > ________________________________________________________________
> >>> > erlang-questions (at) erlang.org mailing list.
> >>> > See http://www.erlang.org/faq.html
> >>> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
> >>> >
> >>>
> >>> --
> >>> ------------------------------------------------------------------------
> >>> Anthony Molinaro <anthonym@REDACTED>
> >>
> >> --
> >> ------------------------------------------------------------------------
> >> Anthony Molinaro <anthonym@REDACTED>
> >>
> >> ________________________________________________________________
> >> erlang-questions (at) erlang.org mailing list.
> >> See http://www.erlang.org/faq.html
> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
> >>
> >>
> >
> >
> >
> > --
> > Andrey Popp
> >
> > phone: +7 911 740 24 91
> > e-mail: 8mayday@REDACTED
> >
> > ________________________________________________________________
> > erlang-questions (at) erlang.org mailing list.
> > See http://www.erlang.org/faq.html
> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
> >
> >
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
--
------------------------------------------------------------------------
Anthony Molinaro <anthonym@REDACTED>
More information about the erlang-questions
mailing list