[erlang-bugs] httpc messing with headers

eurekafag eurekafag@REDACTED
Thu Oct 27 16:14:02 CEST 2011


I wrote a simple multiprocess gallery downloader on Erlang:
http://pastebin.com/bY3cVceH It implements workers pool which get jobs
and save files using httpc:request. If I run 10 or more processes
after some files downloaded I'm getting errors like this:

** Reason for termination ==
** {bad_return_value,{error,{invalid_version,"Ranges:"}}}

The string may be anything else but it always looks like a part of
HTTP header. Sometimes it's a today date, just an empty string [], a
block of data or even "st-modified:" ("Last-modified:" header part).
The easiest way to get a bunch of those errors is to start main/1 like
this:

galdl:main(["http://ru.fishki.net/picsw/042011/13/bonus/kostumi/~3..0w.jpg",
"1", "160", "10"]).

Arguments are strings for starting the program with erl -run. First
20-30 pictures are downloaded ok but then errors come up. I tried
limiting keep-alive, sessions and pipelining to no avail. Looking
tcpdump reports I've found that headers of those files are in the
middle of packet not in the beginning. That's why I think it's
something wrong with persistent connections handling. Googling
revealed almost nothing (a couple of reports in 2009 and a patch which
I already have in R14B03 and an advice to use ibrowse).



More information about the erlang-bugs mailing list