<div dir="ltr"><div><div><div><div>Hi list,<br><br></div>I'm debugging an issue with concurrent downloads and, first of all,<br>i need help understanding how httpc pipelining works.<br><br></div>If i execute the following snippet in Erlang 17.5, this is what i get:<br></div></div><div><div><div><div><br></div><div>--- SNIP --- SNIP --- SNIP ---<br>Erlang/OTP 17 [erts-6.4] [source] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false]<br><br>Eshell V6.4  (abort with ^G)<br>1> inets:start().<br>ok<br>2> httpc:set_options([<br>2>                    {max_keep_alive_length, 2}<br>2>                    ,{keep_alive_timeout, 120000}<br>2>                    ,{max_pipeline_length, 2}<br>2>                    ,{pipeline_timeout, 120000}<br>2>                    ,{max_sessions, 2}<br>2>                   ]).<br>ok<br>3> DownloadFun = fun(Id, Url) -><br>3>                       fun() -><br>3>                               Res = httpc:request(<br>3>                                       get, {Url, []},<br>3>                                       [{version, "HTTP/1.1"}, {connect_timeout, 5000}, {timeout, 300000}],<br>3>                                       [{sync, true},<br>3>                                        {full_result, false},<br>3>                                        {stream, "/tmp/test_file" ++ integer_to_list(Id)}]),<br>3>                               io:format(user, "Process ~p (~p) finished @~p with result ~p, sessions ~p~n", [Id, self(), erlang:now(), Res, httpc:which_sessions()])<br>3>                       end<br>3>               end.<br>#Fun<erl_eval.12.90072148><br>4> erlang:now().<br>{1443,30436,547744}<br>5> [spawn(DownloadFun(Id, "<a href="http://www.erlang.org/download/otp_src_18.1.tar.gz">http://www.erlang.org/download/otp_src_18.1.tar.gz</a>")) || Id <- lists:seq(1,3)].<br>[<0.50.0>,<0.51.0>,<0.52.0>]<br>6> Process 2 (<0.51.0>) finished @{1443,30519,307502} with result {ok,<br>                                                                saved_to_file}, sessions {[{session,<br>                                                                                            {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br>                                                                                              80},<br>                                                                                             <0.55.0>},<br>                                                                                            false,<br>                                                                                            http,<br>                                                                                            #Port<0.725>,<br>                                                                                            ip_comm,<br>                                                                                            1,<br>                                                                                            pipeline,<br>                                                                                            false}],<br>                                                                                          [],<br>                                                                                          []}<br>Process 1 (<0.50.0>) finished @{1443,30526,429106} with result {ok,<br>                                                                saved_to_file}, sessions {[{session,<br>                                                                                            {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br>                                                                                              80},<br>                                                                                             <0.54.0>},<br>                                                                                            false,<br>                                                                                            http,<br>                                                                                            #Port<0.723>,<br>                                                                                            ip_comm,<br>                                                                                            0,<br>                                                                                            pipeline,<br>                                                                                            true},<br>                                                                                           {session,<br>                                                                                            {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br>                                                                                              80},<br>                                                                                             <0.55.0>},<br>                                                                                            false,<br>                                                                                            http,<br>                                                                                            #Port<0.725>,<br>                                                                                            ip_comm,<br>                                                                                            0,<br>                                                                                            pipeline,<br>                                                                                            true}],<br>                                                                                          [],<br>                                                                                          []}<br>Process 3 (<0.52.0>) finished @{1443,30526,490409} with result {ok,<br>                                                                saved_to_file}, sessions {[{session,<br>                                                                                            {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br>                                                                                              80},<br>                                                                                             <0.54.0>},<br>                                                                                            false,<br>                                                                                            http,<br>                                                                                            #Port<0.723>,<br>                                                                                            ip_comm,<br>                                                                                            0,<br>                                                                                            pipeline,<br>                                                                                            true},<br>                                                                                           {session,<br>                                                                                            {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br>                                                                                              80},<br>                                                                                             <0.55.0>},<br>                                                                                            false,<br>                                                                                            http,<br>                                                                                            #Port<0.725>,<br>                                                                                            ip_comm,<br>                                                                                            0,<br>                                                                                            pipeline,<br>                                                                                            true},<br>                                                                                           {session,<br>                                                                                            {{"<a href="http://www.erlang.org">www.erlang.org</a>",<br>                                                                                              80},<br>                                                                                             <0.56.0>},<br>                                                                                            false,<br>                                                                                            http,<br>                                                                                            #Port<0.724>,<br>                                                                                            ip_comm,<br>                                                                                            0,<br>                                                                                            pipeline,<br>                                                                                            true}],<br>                                                                                          [],<br>                                                                                          []}<br><br>6><br>--- SNIP --- SNIP --- SNIP ---<br><br>Given the options i'm setting with httpc:set_options/1, i would expect<br>debugging function httpc:which_sessions/0 to never return 3 open connections<br>to the same host. I would expect 2 connections no matter if the server supports<br>pipelining or not (if not, i guess that one of the three downloads would take longer).<br><br>Another weird thing is that httpc:which_sessions/0 only adds the connection to the list<br>after the resource is downloaded but maybe it is just its expected behaviour.<br></div><div><br></div><div>I'd appreciate any comment that helps me understanding how all this stuff works.<br></div><div>Maybe then, i could properly debug my issue or maybe report some bug :)<br clear="all"></div><div><br></div><div>Regards,<br></div><div><br>-- <br><div class="gmail_signature">Alejandro Vazquez Fente</div>
</div></div></div></div></div>