[erlang-questions] inets httpc pipelining doubt

[Alex Vazquez]

Hi list,

I'm debugging an issue with concurrent downloads and, first of all,
i need help understanding how httpc pipelining works.

If i execute the following snippet in Erlang 17.5, this is what i get:

--- SNIP --- SNIP --- SNIP ---
Erlang/OTP 17 [erts-6.4] [source] [smp:2:2] [async-threads:10] [hipe]
[kernel-poll:false]

Eshell V6.4  (abort with ^G)
1> inets:start().
ok
2> httpc:set_options([
2>                    {max_keep_alive_length, 2}
2>                    ,{keep_alive_timeout, 120000}
2>                    ,{max_pipeline_length, 2}
2>                    ,{pipeline_timeout, 120000}
2>                    ,{max_sessions, 2}
2>                   ]).
ok
3> DownloadFun = fun(Id, Url) ->
3>                       fun() ->
3>                               Res = httpc:request(
3>                                       get, {Url, []},
3>                                       [{version, "HTTP/1.1"},
{connect_timeout, 5000}, {timeout, 300000}],
3>                                       [{sync, true},
3>                                        {full_result, false},
3>                                        {stream, "/tmp/test_file" ++
integer_to_list(Id)}]),
3>                               io:format(user, "Process ~p (~p) finished
@~p with result ~p, sessions ~p~n", [Id, self(), erlang:now(), Res,
httpc:which_sessions()])
3>                       end
3>               end.
#Fun<erl_eval.12.90072148>
4> erlang:now().
{1443,30436,547744}
5> [spawn(DownloadFun(Id, "
http://www.erlang.org/download/otp_src_18.1.tar.gz")) || Id <-
lists:seq(1,3)].
[<0.50.0>,<0.51.0>,<0.52.0>]
6> Process 2 (<0.51.0>) finished @{1443,30519,307502} with result {ok,

saved_to_file}, sessions {[{session,

{{"www.erlang.org",

80},

<0.55.0>},

false,

http,

#Port<0.725>,

ip_comm,

1,

pipeline,

false}],

[],

[]}
Process 1 (<0.50.0>) finished @{1443,30526,429106} with result {ok,

saved_to_file}, sessions {[{session,

{{"www.erlang.org",

80},

<0.54.0>},

false,

http,

#Port<0.723>,

ip_comm,

0,

pipeline,

true},

{session,

{{"www.erlang.org",

80},

<0.55.0>},

false,

http,

#Port<0.725>,

ip_comm,

0,

pipeline,

true}],

[],

[]}
Process 3 (<0.52.0>) finished @{1443,30526,490409} with result {ok,

saved_to_file}, sessions {[{session,

{{"www.erlang.org",

80},

<0.54.0>},

false,

http,

#Port<0.723>,

ip_comm,

0,

pipeline,

true},

{session,

{{"www.erlang.org",

80},

<0.55.0>},

false,

http,

#Port<0.725>,

ip_comm,

0,

pipeline,

true},

{session,

{{"www.erlang.org",

80},

<0.56.0>},

false,

http,

#Port<0.724>,

ip_comm,

0,

pipeline,

true}],

[],

[]}

6>
--- SNIP --- SNIP --- SNIP ---

Given the options i'm setting with httpc:set_options/1, i would expect
debugging function httpc:which_sessions/0 to never return 3 open connections
to the same host. I would expect 2 connections no matter if the server
supports
pipelining or not (if not, i guess that one of the three downloads would
take longer).

Another weird thing is that httpc:which_sessions/0 only adds the connection
to the list
after the resource is downloaded but maybe it is just its expected
behaviour.

I'd appreciate any comment that helps me understanding how all this stuff
works.
Maybe then, i could properly debug my issue or maybe report some bug :)

Regards,

-- 
Alejandro Vazquez Fente
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150923/300d61c6/attachment.htm>