[erlang-questions] Parsing custom HTTP headers with inet, [{active, false},{packet, http_bin}, ...]

[Tony Rogvall]

On 28 aug 2010, at 12.46, zabrane Mikael wrote:

> Hi Tony,
> 
> 2010/8/28 Tony Rogvall <tony@REDACTED>:
>> What  version of OTP are you using?
> 
> I'm under OSX Snow Leopard:
> $ erl
> Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0]
> [kernel-poll:false]
> 
> Eshell V5.7.5  (abort with ^G)
> 1>
> 
Same as me.

> 
>> 
> 
> Hmm ... I'm pretty sure that "curl" sent the headers correctly. I can
> see them with debug "on":
> 
> $ curl -v -D - -x localhost:9876 -H 'X-ARCHWEB-RESOURCE-UUID:
> 123456789' --url http://www.foo.com
> * About to connect() to proxy 127.0.0.1 port 8080 (#0)
> *   Trying 127.0.0.1...   % Total    % Received % Xferd  Average Speed

Hmm why is it saying 8080 ? you wrote 9876?

My line:
curl -v -D - -x localhost:8888 -H 'X-ARCHWEB-RESOURCE-UUID:123456789' --url http://www.foo.com

* About to connect() to proxy localhost port 8888 (#0)
*   Trying ::1... Connection refused
*   Trying fe80::1... Connection refused
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8888 (#0)
> GET http://www.foo.com HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: www.foo.com
> Accept: */*
> Proxy-Connection: Keep-Alive
> X-ARCHWEB-RESOURCE-UUID:123456789
> 

The small test program I used worked fine:

BTW The reason you get mixed cases on some (size <= 20) header names is because the packet parser 
normalize the input to the "normal" case usage (as in Content-Length). The header names
are case insensitive anyway, so you must handle it.


/Tony


-module(http_test).

-compile(export_all).

start() ->
    {ok,L} = gen_tcp:listen(8888, [{active,false},{reuseaddr,true},
				   {packet,http}]),
    accept(L).

accept(L) ->
    case gen_tcp:accept(L) of
	{ok, S} ->
	    loop(S),
	    gen_tcp:close(S),
	    accept(L);
	Error ->
	    io:format("accept: ~p\n", [Error]),
	    accept(L)
    end.


loop(S) ->
    inet:setopts(S, [{active,once}]),
    receive
	{tcp_closed, S} ->
	    io:format("closed\n", []),
	    ok;
	Msg ->
	    io:format("Http: ~p\n", [Msg]),
	    loop(S)
    end.



>  Time    Time     Time  Current
>                                 Dload  Upload   Total   Spent    Left  Speed
> ^M  0     0    0     0    0     0      0      0 --:--:-- --:--:--
> --:--:--     0connected
> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>> GET http://www.lemonde.fr HTTP/1.0
>> User-Agent: curl/7.20.0 (i386-apple-darwin10.3.0) libcurl/7.20.0 OpenSSL/0.9.8o zlib/1.2.5 libidn/1.19
>> Host: www.foo.com
>> Accept: */*
>> Proxy-Connection: Keep-Alive
>> X-WEBARCH-RESOURCE-UUID: 123456789
> ...
> 
> -- 
> Regards
> Zabrane
> 
> 
>> 
>> /Tony
>> 
>> 
>> On 28 aug 2010, at 12.11, zabrane Mikael wrote:
>> 
>>> Hi,
>>> 
>>> I'm facing a strange problem to parse custom HTTP headers in my server
>>> code which looks like:
>>> 
>>> {ok, ListenSocket} = gen_tcp:listen(9876, [binary, inet, {active,
>>> false},  {packet, http_bin}, {reuseaddr, true}]),
>>> ...
>>> headers(Socket) ->
>>>    ok = inet:setopts(Socket, [{active, once}]),
>>>    receive
>>>         ...
>>>        {http, _, {http_header, _, Header, _, Val}} ->
>>>            io:format("---> The header is: ~p: ~p~n", [Header, Val]),
>>> 
>>> Trying to send the following custom header "X-ARCHWEB-RESOURCE-UUID"
>>> works fine, and the custom header appears
>>> correctly in my server code:
>>> $ curl -x localhost:9876 -H 'X-ARCHWEB-RESOURCE-UUID: 123456789' --url
>>> http://www.foo.com
>>> 
>>> 1> ...
>>> ---> The header is: <<"X-ARCHWEB-RESOURCE-UUID">>: <<"123456789">>
>>> 
>>> But changing a bit the custom header name to "X-WEBARCH-RESOURCE-UUID"
>>> doesn't work
>>> and the server don't even print it out (i.e nothing is received).
>>> 
>>> Now, trying with "X-WEBARCH-RESOURC-UUID" (notice the missing "E" at
>>> the end of "RESOURC") works fine:
>>> 
>>> 2> ...
>>> ---> The header is: <<"X-WEBARCH-RESOURC-UUID">>: <<"123456789">>
>>> 
>>> Even bad, trying with "X-WEBARCH-RESOURCE" completely alter the
>>> letters cases (first letter is capital letter and the rest is
>>> lowercase):
>>> 
>>> 3>
>>> ---> The header is: <<"X-Webarch-Resource">>: <<"123456789">>
>>> 
>>> Could someone tell me what am I doing wrong?
>>> Is there any option to tell "inet" to not alter the headers at all?
>>> 
>>> --
>>> Regards
>>> Zabrane
>>> 
>>> ________________________________________________________________
>>> erlang-questions (at) erlang.org mailing list.
>>> See http://www.erlang.org/faq.html
>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>> 
>> 
>> 
> 
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>