[erlang-bugs] httpc messing with headers

[jared.flatow@REDACTED]

We are having the exact same issue currently. At first I was using ibrowse, due to Google saying that httpc is basically unusable, but found that ibrowse had even more serious issues with process leaks. After switching to httpc, things work much better, except for this issue (its not a showstopper since we anyway retry errors), and a memory leak that I can't track down. I'm wondering now though if this might actually be related to the memory leak?

jared

On Oct 27, 2011, at 7:14 AM, ext eurekafag wrote:

> I wrote a simple multiprocess gallery downloader on Erlang:
> http://pastebin.com/bY3cVceH It implements workers pool which get jobs
> and save files using httpc:request. If I run 10 or more processes
> after some files downloaded I'm getting errors like this:
> 
> ** Reason for termination ==
> ** {bad_return_value,{error,{invalid_version,"Ranges:"}}}
> 
> The string may be anything else but it always looks like a part of
> HTTP header. Sometimes it's a today date, just an empty string [], a
> block of data or even "st-modified:" ("Last-modified:" header part).
> The easiest way to get a bunch of those errors is to start main/1 like
> this:
> 
> galdl:main(["http://ru.fishki.net/picsw/042011/13/bonus/kostumi/~3..0w.jpg",
> "1", "160", "10"]).
> 
> Arguments are strings for starting the program with erl -run. First
> 20-30 pictures are downloaded ok but then errors come up. I tried
> limiting keep-alive, sessions and pipelining to no avail. Looking
> tcpdump reports I've found that headers of those files are in the
> middle of packet not in the beginning. That's why I think it's
> something wrong with persistent connections handling. Googling
> revealed almost nothing (a couple of reports in 2009 and a patch which
> I already have in R14B03 and an advice to use ibrowse).
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://erlang.org/mailman/listinfo/erlang-bugs