[erlang-questions] socket_closed_remotely errors with httpc

[jim rosenblum]

In http://erlang.org/pipermail/erlang-questions/2014-February/077788.html and
elsewhere, users observed that httpc would occasionally encounter
`{error,socket_closed_remotely}` when querying a web service of some sort.
The workaround was to use https 1.0, as opposed to 1.1, which is somewhat of
an unsatisfying resolution.

I have this (I believe) same problem and have tried to create a minimal
example that reproduces the issue using a public url to post a simple JSON
body (my 'real' code posts data to a ChicagoBoss server). I am hoping that
someone more qualified than I can take this bit of “research” and maybe
figure out this bug? Alternatively, if someone can nudge me in the right
direction, I *might* be able to take this further…

My code is as follows:


-module(httpc_bug).

-export([test/0]).


test()->

    httpc:set_options([{max_keep_alive_length, 0},

                       {max_pipeline_length, 0},

                       {max_sessions, 0}]),

    test_put(0).


test_put(N) ->

    Url = "https://posttestserver.com/post.php",

    ContentType= "application/json",

    Body = <<"{\"type\":\"body type\",\"value\":2}">>,

    Header = [],

    HttpOptions = [],

    Options = [],

    case httpc:request(post, {Url, Header, ContentType, Body}, HttpOptions,
Options) of

        {ok, {{_V, _Cd, _R}, _RespHeader, B }} ->

                io:format("~p[~p]:~p~n”, [self(), N, B]),

                test_put(N+1);

        {error, Reason} ->

                io:format("~p: died after ~p iterations with: ~p. ~n.",

                        [self(), N, Reason])

        end.


The problem manifests itself regardless of the options (httpc:setoptions/1)
used - or not used. I chose the ones that I did to try to eliminate
code-paths involved.

A typical run looks like this:


Erlang/OTP 17 [erts-6.2] [source-aaaefb3] [64-bit] [smp:8:8]
[async-threads:10] [hipe] [kernel-poll:false]


Eshell V6.2  (abort with ^G)

1> inets:start().

ok

2> ssl:start().

ok

3> code:load_file(httpc_bug).

{module,httpc_bug}

4> httpc_bug:test(1).

[<0.15201.0>]

<0.15201.0>[0]:"Successfully dumped 0 post variables.\nView it at
http://www.posttestserver.com/data/2015/03/13/19.21.45157935823\nPost body
was 30 chars long."

<0.15201.0>[1]:"Successfully dumped 0 post variables.\nView it at
http://www.posttestserver.com/data/2015/03/13/19.21.451784952425\nPost body
was 30 chars long."

. SNIP

<0.15201.0>: died after 102 iterations with: socket_closed_remotely.


I just did five runs, four died in under 410 iterations, and the fifth took
809 iterations.


By examining the contents of the http_manager__session_db before each call
to the httpc:request, I am pretty sure that the flow goes:


httpc_manager.erl line 759 — within a handle_request clause

handle_request(Request, State = #state{options = Options}) ->

    NewRequest = handle_cookies(generate_request_id(Request), State),

    SessionType = session_type(Options),

    case select_session(Request#request.method,

                        Request#request.address,

                         Request#request.scheme, SessionType, State) of

    {ok, HandlerPid} ->

        *pipeline_or_keep_alive(NewRequest, HandlerPid, State);*

        . SNIP



To

httpc_handler.erl line 323 — within a handle_call where keep_alive requests
are

handled.


I don’t have the sophistication to take this any further. Is this enough
information to enable a more qualified individual to pursue this issue.


Thoughts much appreciated.


JR0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150313/b45dcdee/attachment.htm>