[erlang-bugs] httpc messing with headers
Thu Oct 27 16:14:02 CEST 2011
I wrote a simple multiprocess gallery downloader on Erlang:
http://pastebin.com/bY3cVceH It implements workers pool which get jobs
and save files using httpc:request. If I run 10 or more processes
after some files downloaded I'm getting errors like this:
** Reason for termination ==
The string may be anything else but it always looks like a part of
HTTP header. Sometimes it's a today date, just an empty string , a
block of data or even "st-modified:" ("Last-modified:" header part).
The easiest way to get a bunch of those errors is to start main/1 like
"1", "160", "10"]).
Arguments are strings for starting the program with erl -run. First
20-30 pictures are downloaded ok but then errors come up. I tried
limiting keep-alive, sessions and pipelining to no avail. Looking
tcpdump reports I've found that headers of those files are in the
middle of packet not in the beginning. That's why I think it's
something wrong with persistent connections handling. Googling
revealed almost nothing (a couple of reports in 2009 and a patch which
I already have in R14B03 and an advice to use ibrowse).
More information about the erlang-bugs