[erlang-bugs] httpc messing with headers

[jared.flatow@REDACTED]

Hi Tino,

Yes I looked at it, but after my experience with ibrowse, and the fact that there is even less documentation for it than httpc, I was a little uncomfortable. Also, its difficult to see who has been using it and maintaining it. I'm trying it out now and it seems to work pretty nicely though. I agree with Eureka it would be good if the version in the std library worked (I know you probably agree). Anyway, lhttpc indeed appears to be a good solution, I'll follow up here if I do run into any issues :)

Thanks!
jared

On Oct 27, 2011, at 10:25 AM, ext Tino Breddin wrote:

> Hi Jared,
> 
> I've not come across the error in question. But I'm wondering whether you've looked at using lhttpc [1] instead of ibrowse and httpc, and if so why you chose to not use it.
> 
> Cheers,
> Tino
> 
> [1] https://github.com/esl/lhttpc
> 
> On Oct 27, 2011, at 6:43 PM, <jared.flatow@REDACTED> wrote:
> 
>> We are having the exact same issue currently. At first I was using ibrowse, due to Google saying that httpc is basically unusable, but found that ibrowse had even more serious issues with process leaks. After switching to httpc, things work much better, except for this issue (its not a showstopper since we anyway retry errors), and a memory leak that I can't track down. I'm wondering now though if this might actually be related to the memory leak?
>> 
>> jared
>> 
>> On Oct 27, 2011, at 7:14 AM, ext eurekafag wrote:
>> 
>>> I wrote a simple multiprocess gallery downloader on Erlang:
>>> http://pastebin.com/bY3cVceH It implements workers pool which get jobs
>>> and save files using httpc:request. If I run 10 or more processes
>>> after some files downloaded I'm getting errors like this:
>>> 
>>> ** Reason for termination ==
>>> ** {bad_return_value,{error,{invalid_version,"Ranges:"}}}
>>> 
>>> The string may be anything else but it always looks like a part of
>>> HTTP header. Sometimes it's a today date, just an empty string [], a
>>> block of data or even "st-modified:" ("Last-modified:" header part).
>>> The easiest way to get a bunch of those errors is to start main/1 like
>>> this:
>>> 
>>> galdl:main(["http://ru.fishki.net/picsw/042011/13/bonus/kostumi/~3..0w.jpg",
>>> "1", "160", "10"]).
>>> 
>>> Arguments are strings for starting the program with erl -run. First
>>> 20-30 pictures are downloaded ok but then errors come up. I tried
>>> limiting keep-alive, sessions and pipelining to no avail. Looking
>>> tcpdump reports I've found that headers of those files are in the
>>> middle of packet not in the beginning. That's why I think it's
>>> something wrong with persistent connections handling. Googling
>>> revealed almost nothing (a couple of reports in 2009 and a patch which
>>> I already have in R14B03 and an advice to use ibrowse).
>>> _______________________________________________
>>> erlang-bugs mailing list
>>> erlang-bugs@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-bugs
>> 
>> _______________________________________________
>> erlang-bugs mailing list
>> erlang-bugs@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-bugs
>