[erlang-questions] gen_tcp very slow to fetch data

[Colm Dougan]

On Mon, Nov 23, 2009 at 11:58 AM, zabrane Mikael <zabrane3@REDACTED> wrote:
>>
>> Your assumption that the initial gen_tcp:recv will give you a binary
>> which always includes the full request headers is not correct.  If you
>> are going down that road you would need to write code to check for the
>> the header terminator and if it isn't present then buffer and read
>> more until you have the terminator.
>
> Ok. Finding headers terminator (teh double CRLF) isn't difficult.
> What's difficult for me is to find the request end?
> How can I know that I've read all data in my request (GET or POST)?

erlang doesn't provide any magic way to read the request body (in the
way it does for headers).   You have to write that code yourself.  You
can look at either the yaws or mochiweb sources for straightforward
erlang implementations of this.

>>  It isn't hard but it can be messy.
>
> Joe's code helped me a lot with his reentrant parser. Thanks by the way.
>>
>> Most erlang http servers these days use the {packet, http} socket
>> option which is HTTP aware and which will read/parse headers one by
>> one for you and will also indicate the end of the headers.
>
> Lev Walkin introduced an example of his web server called yucan which uses
> {packer, http_bin}:
> http://lionet.livejournal.com/42016.html?thread=774432
> Could someone explain the difference?

With the "http" option you get the header key and value returned to
you as strings, apart from well known header names which are returned
as atoms.  With "http_bin" the same applies, but where there were
strings you get erlang binaries instead.

Colm