[erlang-questions] Re: Frequent crashes in inets http client (R12B-5)

[Oscar Hellström]

Hi All,

As usual, I type too much or too little. The short answer would be:
ibrowse also have issues (see my previous email), but not the blocking
behaviour of inets http client. It can however be a real CPU and memory
drag, and in some cases the timeout handling is questionable.

Best regards

Steve Davis wrote:
> Hi Oscar,
>
> Do you happen to know whether ibrowse suffers the same limitations?
>
> regs,
> /s
>
> On Jun 5, 9:42 am, Oscar Hellström <os...@REDACTED>
> wrote:
>   
>> Not to start a flame war, but I would stay away from the inets http
>> client if I were trying to build something serious. You can find my
>> reasons here:http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:43806:200905:gocblgddep...
>>
>> Best regards
>>
>>
>>
>>
>>
>> Chris Newcombe wrote:
>>     
>>> Is there a patch for the following issue?
>>>       
>>> It was reported a while ago:
>>> http://groups.google.com/group/erlang-programming/browse_thread/threa...
>>> I didn't see any replies)
>>>       
>>> Here's a bit more detail:
>>>       
>>> httpc_handler is crashing with
>>>       
>>>        {badrecord,request}
>>>       
>>> (BTW it would be great if badrecord errors also contained the incorrect
>>> term, not just the name of the expected record type)
>>>       
>>> It’s crashing in
>>>       
>>>        httpc_handler,handle_info,2
>>>       
>>> The last message received by the gen_server is
>>>       
>>>       {timeout,#Ref<0.0.0.9038>}
>>>       
>>> The gen_server #state is
>>>       
>>> {state,undefined,{tcp_session,{{"my-test-url",8080},<0.709.0>},false,http,# Port<0.1351>,1},undefined,undefined,undefined,undefined,{[],[]},pipeline,[# Ref<0.0.0.5834>],nolimit,nolimit,{options,{undefined,[]},20000,1,100,disabl ed,enabled,false},{timers,[],#Ref<0.0.0.19293>}
>>>       
>>> I think the relevant element is the first one (request).
>>>       
>>> i.e. request == undefined
>>>       
>>> Given the message, it seems almost certain that the crash is in the second
>>> timeout clause of handle_info,
>>>       
>>> (marked below with ***).
>>>       
>>> This clause will fire even if request == undefined, but will try to use
>>> Request#request.from, which crashes with {badrecord,request}
>>>       
>>>        %%% Timeouts
>>>       
>>>        %% Internaly, to a request handling process, a request time out is
>>>       
>>>        %% seen as a canceld request.
>>>       
>>>        handle_info({timeout, RequestId}, State =
>>>       
>>>                    #state{request = Request = #request{id = RequestId}}) ->
>>>       
>>>            httpc_response:send(Request#request.from,
>>>       
>>>                          httpc_response:error(Request,timeout)),
>>>       
>>>            {stop, normal,
>>>       
>>>             State#state{canceled = [RequestId | State#state.canceled],
>>>       
>>>                         request = Request#request{from = answer_sent}}};
>>>       
>>> ***    handle_info({timeout, RequestId}, State = #state{request = Request})
>>> ->
>>>       
>>>            httpc_response:send(Request#request.from,
>>>       
>>>                               httpc_response:error(Request,timeout)),
>>>       
>>>            {noreply, State#state{canceled = [RequestId |
>>> State#state.canceled]}};
>>>       
>>>        handle_info(timeout_pipeline, State = #state{request = undefined}) ->
>>>       
>>>            {stop, normal, State};
>>>       
>>> It looks like State#state.request is being set to undefined without
>>> cancelling an in-progress request timer.
>>>       
>>> I've only glanced at the code, but both of the following clauses appear to
>>> do that.
>>>       
>>> (But it could easily be something else.)
>>>       
>>>         %% On a redirect or retry the current request becomes
>>>       
>>>         %% obsolete and the manager will create a new request
>>>       
>>>         %% with the same id as the current.
>>>       
>>>         {redirect, NewRequest, Data}->
>>>       
>>>             ok = httpc_manager:redirect_request(NewRequest, ProfileName),
>>>       
>>>             handle_pipeline(State#state{request = undefined}, Data);
>>>       
>>>         {retry, TimeNewRequest, Data}->
>>>       
>>>             ok = httpc_manager:retry_request(TimeNewRequest, ProfileName),
>>>       
>>>             handle_pipeline(State#state{request = undefined}, Data);
>>>       
>>> thanks,
>>>       
>>> Chris
>>>       
>> --
>> Oscar Hellström, os...@REDACTED
>> Office: +44 20 7655 0337
>> Mobile: +44 798 45 44 773
>> Erlang Training and Consulting Ltdhttp://www.erlang-consulting.com/
>>
>> ________________________________________________________________
>> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html
>> erlang-questions (at) erlang.org
>>     
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>   


-- 
Oscar Hellström, oscar@REDACTED
Office: +44 20 7655 0337
Mobile: +44 798 45 44 773
Erlang Training and Consulting Ltd
http://www.erlang-consulting.com/