[erlang-questions] httpc manager crashes

[Ingela Andin]

Hi!

A httpc-profile could be view as a "browser core", it saves options
that you want to use for all requests such as proxy-options,
it handles cookies, it dispatches requests for pipelining or
persistent HTTP-connections queues, it could even do caching but that
is not implemented at the moment.  There is no hard limit on how many
profiles you can use, a profile is mainly a new dispatcher process.
300 erlang-processes is no problem at all, but starting a new profile
for every request I think is a bit of an odd strategy.  Why do you do
that? The manager process will spawn a new request-handler process for
each request unless it can be pipelined, or if pipelining is not
enabled by setting the pipeline timeout and there is a persistent
connection that can be used to queue the request on. Length of
pipeline and persistent
connection-queues are configurable. After the request has been
dispatched to the handler process that process will handle the request
and send the answer to the client without any synchronizing with the
manager process. The manager process should not be very busy so it
should not be a big bottleneck and if it gets too busy we might have
to change its implementation.

We also think we know what is causing your problem and we will fix it
as soon as possible.

Regards Ingela Erlang/OTP team - Ericsson AB


2011/4/28 Suma Shivaprasad <sumasai.shivaprasad@REDACTED>:
> Hi,
>
> We are using inets - 5.5.1 and HTTP/1.1.  We are creating a new httpc
> profile for every request that we make and around 300 concurrent requests
> were running at the same time . Is this fine?  How many maximum profiles has
> httpc been tested with for concurrent use?
>
> Thanks
> Suma
>
> On Thu, Apr 28, 2011 at 2:53 PM, Ingela Andin <ingela.andin@REDACTED>
> wrote:
>>
>> Hi!
>>
>> What version of inets and Erlang are you using?  What HTTP version
>> HTTP-1.0 ?
>> If there is a bug we will of course fix it.
>>
>> Regards Ingela Erlang/OTP - Ericsson AB
>>
>>
>> 2011/4/27 Suma Shivaprasad <sumasai.shivaprasad@REDACTED>:
>> > We have an app that makes a lot of calls to httpc and we are seeing the
>> > following error in sasl logs quite often.
>> >
>> > CRASH REPORT==== 27-Apr-2011::05:21:24 ===
>> >   crasher:
>> >     initial call: httpc_manager:init/1
>> >     pid: <0.10297.0>
>> >     registered_name: httpc_manager_profile_8261
>> >     exception exit: {{case_clause,
>> >                          [{handler_info,#Ref<0.0.398.166371>,undefined,
>> >                               <0.12971.248>,undefined,operational}]},
>> >                      [{httpc_manager,handle_connect_and_send,5},
>> >                       {httpc_manager,handle_info,2},
>> >                       {gen_server,handle_msg,5},
>> >                       {proc_lib,init_p_do_apply,3}]}
>> >       in function  gen_server:terminate/6
>> >     ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>
>> >
>> >
>> >
>> > and
>> >
>> >
>> > =CRASH REPORT==== 27-Apr-2011::05:30:18 ===
>> >   crasher:
>> >     initial call: httpc_manager:init/1
>> >     pid: <0.22933.12>
>> >     registered_name: httpc_manager_profile_8122
>> >     exception exit: {{case_clause,
>> >
>> > [{handler_info,#Ref<0.0.418.148452>,<0.20608.257>,
>> >                               undefined,<0.15267.257>,initiating}]},
>> >                      [{httpc_manager,handle_connect_and_send,5},
>> >                       {httpc_manager,handle_info,2},
>> >                       {gen_server,handle_msg,5},
>> >                       {proc_lib,init_p_do_apply,3}]}
>> >       in function  gen_server:terminate/6
>> >     ancestors: [httpc_profile_sup,httpc_sup,inets_sup,<0.46.0>]
>> >     messages: [{'EXIT',<0.20574.257>,normal}]
>> >     links: [<0.50.0>,<0.20608.257>]
>> >     dictionary: []
>> >
>> >
>> > Has anyone else seen these errors or have a fix for this?
>> >
>> > Thanks
>> > Suma
>> >
>> >
>> >
>> > _______________________________________________
>> > erlang-questions mailing list
>> > erlang-questions@REDACTED
>> > http://erlang.org/mailman/listinfo/erlang-questions
>> >
>> >
>
>