http:set_options/1, http:request/4

Michael McDaniel erlang@REDACTED
Fri Jan 21 07:31:57 CET 2005

http:request/4 used directly will properly retrieve URL.
http:request/4 through proxy fails.


Linux fangora 2.6.4-52-default #1 Wed Apr 7 02:08:30 UTC 2004 i686 i686 i386 

using erl
Eshell V5.4.3  (abort with ^G)

	 | |
	| |
	| |

hd.erl is running on inside private network.
Squid proxy is running on (aka
on port 3128.


hd(Port, URL) ->
  application:start(inets) ,

  http:set_options([{proxy, { {"", Port},
	    ["", "", "localhost"]} }]) ,

  {URL,  [ {"Referer", URL}, {"Host", ""} ]}, [], []).

%% end hd.erl

1> hd:hd(3128,"").
{ok,{{"HTTP/1.0",400,"Bad Request"}, ...
2> hd:hd(3128,""). 
{ok,{{"HTTP/1.1",200,"OK"}, ...

The problem appears to be that the http:request/4 command in both of
the above invocations create the following get:

GET / HTTP/1.1

This is fine when not using proxy.  When using proxy, the proxy server
is expecting, for example, "GET HTTP/1.1",
so the get request fails.  Follows is the headers trace...

GET / HTTP/1.1

HTTP/1.0 400 Bad Request
Server: squid/3.0-PRE3
Mime-Version: 1.0
Date: Fri, 21 Jan 2005 05:55:23 GMT
Content-Type: text/html
Content-Length: 1436
Expires: Fri, 21 Jan 2005 05:55:23 GMT
X-Squid-Error: ERR_INVALID_URL 0
X-Cache: MISS from
X-Cache-Lookup: NONE from
Via: 1.0 (squid/3.0-PRE3)
Proxy-Connection: close

Follows is headers trace from a succesful browser GET

User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a4) Gecko/20040927
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Proxy-Connection: keep-alive

HTTP/1.0 200 OK
Date: Fri, 21 Jan 2005 06:28:51 GMT
Server: Apache/1.3.9 (Unix)
Last-Modified: Wed, 15 Dec 2004 14:24:04 GMT
ETag: "8d715-1efb-41c04904"
Accept-Ranges: bytes
Content-Length: 7931
Content-Type: text/html
X-Cache: MISS from
X-Cache-Lookup: MISS from
Via: 1.0 (squid/3.0-PRE3)
Proxy-Connection: keep-alive

At first I thought the problem was the TE: (transport encoding) header,
but it is not.  I did the tests with adding the other headers.  The
problem is the initial "GET".  The browser is configured to use the
squid proxy on port 3128.

I tried the following:
1> hd:hd(3128,"").

and the GET is "GET / HTTP/1.1" which is still

Any suggestions are welcome.  I want to be able to use the commands
from my inside machine through proxy on Internet connected machine.

Michael McDaniel
Portland, Oregon, USA

More information about the erlang-questions mailing list