<div dir="ltr"><div><div><div><div>Hi list,<br><br></div>I'm debugging an issue with concurrent downloads and, first of all,<br>i need help understanding how httpc pipelining works.<br><br></div>If i execute the following snippet in Erlang 17.5, this is what i get:<br></div></div><div><div><div><div><br></div><div>--- SNIP --- SNIP --- SNIP ---<br>Erlang/OTP 17 [erts-6.4] [source] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false]<br><br>Eshell V6.4 (abort with ^G)<br>1> inets:start().<br>ok<br>2> httpc:set_options([<br>2> {max_keep_alive_length, 2}<br>2> ,{keep_alive_timeout, 120000}<br>2> ,{max_pipeline_length, 2}<br>2> ,{pipeline_timeout, 120000}<br>2> ,{max_sessions, 2}<br>2> ]).<br>ok<br>3> DownloadFun = fun(Id, Url) -><br>3> fun() -><br>3> Res = httpc:request(<br>3> get, {Url, []},<br>3> [{version, "HTTP/1.1"}, {connect_timeout, 5000}, {timeout, 300000}],<br>3> [{sync, true},<br>3> {full_result, false},<br>3> {stream, "/tmp/test_file" ++ integer_to_list(Id)}]),<br>3> io:format(user, "Process ~p (~p) finished @~p with result ~p, sessions ~p~n", [Id, self(), erlang:now(), Res, httpc:which_sessions()])<br>3> end<br>3> end.<br>#Fun<erl_eval.12.90072148><br>4> erlang:now().<br>{1443,30436,547744}<br>5> [spawn(DownloadFun(Id, "<a href="http://www.erlang.org/download/otp_src_18.1.tar.gz">http://www.erlang.org/download/otp_src_18.1.tar.gz</a>")) || Id <- lists:seq(1,3)].<br>[<0.50.0>,<0.51.0>,<0.52.0>]<br>6> Process 2 (<0.51.0>) finished @{1443,30519,307502} with result {ok,<br> saved_to_file}, sessions {[{session,<br> {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br> 80},<br> <0.55.0>},<br> false,<br> http,<br> #Port<0.725>,<br> ip_comm,<br> 1,<br> pipeline,<br> false}],<br> [],<br> []}<br>Process 1 (<0.50.0>) finished @{1443,30526,429106} with result {ok,<br> saved_to_file}, sessions {[{session,<br> {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br> 80},<br> <0.54.0>},<br> false,<br> http,<br> #Port<0.723>,<br> ip_comm,<br> 0,<br> pipeline,<br> true},<br> {session,<br> {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br> 80},<br> <0.55.0>},<br> false,<br> http,<br> #Port<0.725>,<br> ip_comm,<br> 0,<br> pipeline,<br> true}],<br> [],<br> []}<br>Process 3 (<0.52.0>) finished @{1443,30526,490409} with result {ok,<br> saved_to_file}, sessions {[{session,<br> {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br> 80},<br> <0.54.0>},<br> false,<br> http,<br> #Port<0.723>,<br> ip_comm,<br> 0,<br> pipeline,<br> true},<br> {session,<br> {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br> 80},<br> <0.55.0>},<br> false,<br> http,<br> #Port<0.725>,<br> ip_comm,<br> 0,<br> pipeline,<br> true},<br> {session,<br> {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br> 80},<br> <0.56.0>},<br> false,<br> http,<br> #Port<0.724>,<br> ip_comm,<br> 0,<br> pipeline,<br> true}],<br> [],<br> []}<br><br>6><br>--- SNIP --- SNIP --- SNIP ---<br><br>Given the options i'm setting with httpc:set_options/1, i would expect<br>debugging function httpc:which_sessions/0 to never return 3 open connections<br>to the same host. I would expect 2 connections no matter if the server supports<br>pipelining or not (if not, i guess that one of the three downloads would take longer).<br><br>Another weird thing is that httpc:which_sessions/0 only adds the connection to the list<br>after the resource is downloaded but maybe it is just its expected behaviour.<br></div><div><br></div><div>I'd appreciate any comment that helps me understanding how all this stuff works.<br></div><div>Maybe then, i could properly debug my issue or maybe report some bug :)<br clear="all"></div><div><br></div><div>Regards,<br></div><div><br>-- <br><div class="gmail_signature">Alejandro Vazquez Fente</div>
</div></div></div></div></div>