Helo Ingela<div><br></div><div>I am Suma's colleague. Let me try to give more context to the issue we are facing..</div><div><br></div><div>* Our app make a LOT (in the range of around 300) of http requests concurrently.</div>
<div>* There are two types of requests</div><div><meta charset="utf-8"> 1) POST requests : These require us to go thru a proxy. These can themselves hit about 150 - 200concurrent requests.</div><div> 2) GET requests : Standard http requests.</div>
<div><br></div><div><meta charset="utf-8">As far as the GET requests are concerned, we start some 300 profiles and randomly pick one for use.</div><div><br></div><div>Since we have to send our POSTs via a proxy, at app initialialization time, we start 1 httpc profile. We see a lot of httpc_manager crashes pertaining to this particular profile. I understand that the httpc_manager is supposed to be a pretty light weight process.. not requiring any synchronization with the manager.. Is there a chance that the httpc_manager can crash ? Also, I was wondering, is the Data to be posted also copied to the httpc_managers message queue before its again copied to the handlers message queue before actually sending ?</div>
<div><br></div><div><br></div><div>Regards</div><div>-Arun</div><div> </div><div><br><br><div class="gmail_quote">On Fri, Apr 29, 2011 at 12:49 PM, Ingela Andin <span dir="ltr"><<a href="mailto:ingela@erlang.org">ingela@erlang.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Hi!<br>
<br>
A httpc-profile could be view as a "browser core", it saves options<br>
that you want to use for all requests such as proxy-options,<br>
it handles cookies, it dispatches requests for pipelining or<br>
persistent HTTP-connections queues, it could even do caching but that<br>
is not implemented at the moment. There is no hard limit on how many<br>
profiles you can use, a profile is mainly a new dispatcher process.<br>
300 erlang-processes is no problem at all, but starting a new profile<br>
for every request I think is a bit of an odd strategy. Why do you do<br>
that? The manager process will spawn a new request-handler process for<br>
each request unless it can be pipelined, or if pipelining is not<br>
enabled by setting the pipeline timeout and there is a persistent<br>
connection that can be used to queue the request on. Length of<br>
pipeline and persistent<br>
connection-queues are configurable. After the request has been<br>
dispatched to the handler process that process will handle the request<br>
and send the answer to the client without any synchronizing with the<br>
manager process. The manager process should not be very busy so it<br>
should not be a big bottleneck and if it gets too busy we might have<br>
to change its implementation.<br>
<br>
We also think we know what is causing your problem and we will fix it<br>
as soon as possible.<br>
<br>
Regards Ingela Erlang/OTP team - Ericsson AB<br>
<br>
<br>
2011/4/28 Suma Shivaprasad <<a href="mailto:sumasai.shivaprasad@gmail.com">sumasai.shivaprasad@gmail.com</a>>:<br>
<div><div></div><div class="h5">> Hi,<br>
><br>
> We are using inets - 5.5.1 and HTTP/1.1. We are creating a new httpc<br>
> profile for every request that we make and around 300 concurrent requests<br>
> were running at the same time . Is this fine? How many maximum profiles has<br>
> httpc been tested with for concurrent use?<br>
><br>
> Thanks<br>
> Suma<br>
><br>
> On Thu, Apr 28, 2011 at 2:53 PM, Ingela Andin <<a href="mailto:ingela.andin@gmail.com">ingela.andin@gmail.com</a>><br>
> wrote:<br>
>><br>
>> Hi!<br>
>><br>
>> What version of inets and Erlang are you using? What HTTP version<br>
>> HTTP-1.0 ?<br>
>> If there is a bug we will of course fix it.<br>
>><br>
>> Regards Ingela Erlang/OTP - Ericsson AB<br>
>><br>
>><br>
>> 2011/4/27 Suma Shivaprasad <<a href="mailto:sumasai.shivaprasad@gmail.com">sumasai.shivaprasad@gmail.com</a>>:<br>
>> > We have an app that makes a lot of calls to httpc and we are seeing the<br>
>> > following error in sasl logs quite often.<br>
>> ><br>
>> > CRASH REPORT==== 27-Apr-2011::05:21:24 ===<br>
>> > crasher:<br>
>> > initial call: httpc_manager:init/1<br>
>> > pid: <0.10297.0><br>
>> > registered_name: httpc_manager_profile_8261<br>
>> > exception exit: {{case_clause,<br>
>> > [{handler_info,#Ref<0.0.398.166371>,undefined,<br>
>> > <0.12971.248>,undefined,operational}]},<br>
>> > [{httpc_manager,handle_connect_and_send,5},<br>
>> > {httpc_manager,handle_info,2},<br>
>> > {gen_server,handle_msg,5},<br>
>> > {proc_lib,init_p_do_apply,3}]}<br>
>> > in function gen_server:terminate/6<br>
>> > ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0><br>
>> ><br>
>> ><br>
>> ><br>
>> > and<br>
>> ><br>
>> ><br>
>> > =CRASH REPORT==== 27-Apr-2011::05:30:18 ===<br>
>> > crasher:<br>
>> > initial call: httpc_manager:init/1<br>
>> > pid: <0.22933.12><br>
>> > registered_name: httpc_manager_profile_8122<br>
>> > exception exit: {{case_clause,<br>
>> ><br>
>> > [{handler_info,#Ref<0.0.418.148452>,<0.20608.257>,<br>
>> > undefined,<0.15267.257>,initiating}]},<br>
>> > [{httpc_manager,handle_connect_and_send,5},<br>
>> > {httpc_manager,handle_info,2},<br>
>> > {gen_server,handle_msg,5},<br>
>> > {proc_lib,init_p_do_apply,3}]}<br>
>> > in function gen_server:terminate/6<br>
>> > ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>]<br>
>> > messages: [{'EXIT',<0.20574.257>,normal}]<br>
>> > links: [<0.50.0>,<0.20608.257>]<br>
>> > dictionary: []<br>
>> ><br>
>> ><br>
>> > Has anyone else seen these errors or have a fix for this?<br>
>> ><br>
>> > Thanks<br>
>> > Suma<br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > erlang-questions mailing list<br>
>> > <a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
>> > <a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
>> ><br>
>> ><br>
><br>
><br>
_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
</div></div></blockquote></div><br></div>