[erlang-bugs] httpc messing with headers
eurekafag
eurekafag@REDACTED
Thu Oct 27 19:37:51 CEST 2011
I'm glad I'm not the only one. However, this issue is reproduced only
on my home computer but not on my servers. One of them has much better
speed (about 100 Mbit up/down) than me (about 5.5 Mbit) and it
downloads everything in a second without a problem. Another server has
poor speed (1-2 Mbit) and the program also works fine there. I tried
R14A which is on those servers but that changed nothing. Also, I used
compiled .beam there so it's not a compiler problem, maybe some race
condition as others mentioned. I suggest using tcpdump to narrow the
problem down and find the request which leads to the failure. For now
I have one case (reusing the connection with HTTP response header
placed in the middle of TCP packet) maybe you'll find any other.
Just for completeness sake here's the full error message:
http://pastebin.com/tyYTKjZw
Regarding Tino's suggestion: well, I think it's better to use standart
library if you can. Just because it's standart and it works out of the
box. I'm not asking something advanced, just requesting urls and
saving the data into files. It's just weird that language made for
writing distributed and parallel programs fails processing parallel
plain HTTP requests and gets confused with persistent connections.
It's ok if the library lacks some functions but not if it glitches
2011/10/27 <jared.flatow@REDACTED>:
> We are having the exact same issue currently. At first I was using ibrowse, due to Google saying that httpc is basically unusable, but found that ibrowse had even more serious issues with process leaks. After switching to httpc, things work much better, except for this issue (its not a showstopper since we anyway retry errors), and a memory leak that I can't track down. I'm wondering now though if this might actually be related to the memory leak?
>
> jared
More information about the erlang-bugs
mailing list