Inets HTTP client stability

[Martin Carlson]

I would probably opt at yaws/ibrowse or a simple gen_tcp.
To my knowlage yaws/ibrowse. The java node seems to me
like a really odd choice since Erlang is designed
for concurrency and have excelent network support.

Martin
-------------------------
Martin Carlson
www.erlang-consulting.com

On Thu, 27 Oct 2005, Michael McDaniel wrote:

> On Thu, Oct 27, 2005 at 02:34:21PM +0200, Heinrich Venter wrote:
>> Hi all.
>>
>> I am about to develop an app that will act as an http gateway for some of the
>> services we are running.  For the incoming http requests I plan to use httpd
>> from inets.  For the outgonig leg it would thus make sense to use the http
>> client from inets.
>> My question is wether any one else is using the inets http client for medium to
>> high volume outgoing traffic?  I know of two instances where people chose to
>> create a java node to make http connections, instead of using inets.
>> Is it worth the risk to try the inets solution first, or should I budget the
>> time to make a java node and try to upgrade to inets later?
>> Any one with experience in this regard?
>>
>> The requirements for the http client will be that many connections will be made
>> to a series of speciffic URLs.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> I have successfully been using http:request/4 for doing POSTs for a
> few months now in a production system.  I am very happy with the
> results (and Erlang!).
>
> See code below...
>
> The production system is not supporting heavy traffic (though the customer
> hopes to make more sales!); however, during testing I was very pleased.
>
> The destination server is provided by another company, and their
> testing server could not keep up with my load testing.  At one point I
> had between 800-900 http:request/4s backed up retrying as described
> below.  All the POSTs eventually were successful with a response from the
> test server.  An abbreviated version of the code is below...
>
>
> -define(DEFAULT_TIMEOUT, 2000).
> MyFun(Timeout, Body) ->
>
>  case (catch http:request(post,
>  {"http://example.com/path/" ,
>  [
>     {"Date", httpd_util:rfc1123_date()} ,
>     {"Host", "example.com"} ,
>     {"Accept", "text/xml/html"},
>     {"User-Agent",  "Demo Erlang Client"}
>  ], "text/xml; charset=utf-8", Body },
>     [{keepalive, false}, {nodelay,true}], [])) of
>       {ok, Result}    -> {ok, Result} ;
>       {error, Reason} ->
> 		      Result = null ,
>                      mylogger:log_error(?MODULE, ?LINE,
> 		      'MyFun, http:request', "failed, see /tmp/my.log"),
> 		      {ok, Io} = file:open('/tmp/my.log', [append]) ,
> 	 	      io:fwrite(Io, "~26p, ~p~n",
> 					[erlang:localtime(), Reason]) ,
> 		      file:close(Io) ,
> 		      if TimeOut > 8192000 ->  % milliseconds
> 	 		        timer:sleep(TimeOut/8) ,
> 				?MODULE:MyFun(?DEFAULT_TIMEOUT,Body);
> 			 true ->
> 	 		        timer:sleep(TimeOut) ,
> 				?MODULE:MyFun(2*TimeOut,Body)
> 		      end
>  end ,
>
> 	%%% Do something with Result ...
>
> end.
>
> Of course you could give up eventually but I "know" I'll eventually be
> able to get where I'm POSTing to, hence the endless retries.
>
> You may prefer to spawn rather than serialize the repeat fun calls as
> I have done.
>
> Please anyone mention if there are more accepted/efficient ways of
> accomplishing the above.  I still consider myself very novice with
> Erlang and Erlang idioms.  I have not yet started using OTP yet,
> and still am writing "pure" Erlang.  I have a new project coming up
> and will probably dabble with OTP on that one.
>
> ~Michael
>
>
>
>
>
>
>