[erlang-questions] gen_tcp very slow to fetch data

[Colm Dougan]

On Sat, Nov 21, 2009 at 12:15 AM, zabrane Mikael <zabrane3@REDACTED> wrote:
> Thanks Robert. I just made a quick copy/past for the post and I forgot the
> "~p" ;-)
> So my problem is still there if you can help !!

Your assumption that the initial gen_tcp:recv will give you a binary
which always includes the full request headers is not correct.  If you
are going down that road you would need to write code to check for the
the header terminator and if it isn't present then buffer and read
more until you have the terminator.  It isn't hard but it can be
messy.

Most erlang http servers these days use the {packet, http} socket
option which is HTTP aware and which will read/parse headers one by
one for you and will also indicate the end of the headers.
mochiweb_http has a good example of how to use {packet, http} to read
request headers :

   http://code.google.com/p/mochiweb/source/browse/trunk/src/mochiweb_http.erl

Colm


>
> 2009/11/21 Robert Virding <rvirding@REDACTED>
>
>> The call to io:format is wrong as it is missing an argument to print with
>> ~p. So it will print the "BinData: " leader and then crash. I am guessing
>> you meant it to be:
>>
>> io:fornat("BinData: ~p~n", [BinData]),
>>
>> You should have got a warning from the compiler.
>>
>> Robert
>>
>> 2009/11/21 zabrane Mikael <zabrane3@REDACTED>
>>
>>> Hi List !
>>>
>>> While trying to learn how to write a simple TCP Web Server in Erlang which
>>> only dump what it gets to stdout, I realize that time to time, the HTTP
>>> requests get truncated when reaching the server. My main socket loop looks
>>> like:
>>>
>>> -------------------------------------------
>>> -define(TCP_OPTIONS,[binary, {packet, 0}, {active, false}, {reuseaddr,
>>> true}]).
>>> ...
>>> loop_recv(Socket)
>>>   case gen_tcp:recv(Socket, 0) of
>>>        {ok, BinData} ->
>>>             %% here, I'm assuming that all the HTTP request (Headers +
>>> Body) is in "BinData". Hope I'm right.
>>>             io:format("BinData: ~p~n", []),
>>>             ok;
>>>        NotOK ->
>>>            error_logger:info_report([{"gen_tcp:recv/2", NotOK}]),
>>>            error
>>>    end.
>>> -------------------------------------------
>>>
>>> For some requests, the "io:format" prints a truncated data (in BinData):
>>>
>>> <<"
>>>
>>> http://www.foo.tv/images/v30/LoaderV3.swf?loop=false&quality=high&request=357&HTTP/1.0
>>> \r\nHost:
>>> www.foo.tv\r\nUser-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5;
>>> fr; rv:1.9.0.2) Gecko/2008090512 Firefox/3.0.2\r\nAccept: text/h">>
>>>
>>> As you can see, the request isn't complete "... Accept: text/h".
>>>
>>> Am I doing somthing wrong? How can I fix it please?
>>>
>>> Regards
>>> Zabrane
>>>
>>>
>