[erlang-questions] What lib to use for http requests

[Mark Nijhof]

Hi Drew,

Thanks, especially for the HTTPS links!

-Mark


On Fri, Dec 12, 2014 at 10:18 PM, Drew Varner <drew.varner@REDACTED>
wrote:

> Mark,
>
> If you are communicating to servers via HTTPS, Hackney and other HTTP
> clients allow you to pass options to the underlying SSL/TLS socket,
> including some verification of the peer certificate. For example:
>
>
> https://github.com/talko/httpcbench/blob/master/src/httpcbench_client.erl#L79-L86
>
> Gun has an open issue to address this (
> https://github.com/extend/gun/pull/27), but it is not implemented. If you
> use Gun as your HTTPS client, you’re open to man-in-the-middle attacks.
>
> Do not fall into a false sense of security that any Erlang HTTPS clients
> provide complete protection against man-in-the-middle attacks out of the
> box. You’ll also want to consider cases where your peer certificate was
> revoked by a Certificate Authority. You’d want to know how your HTTPS
> client handles certificates when their revocation data has been published
> via a CRL or OCSP. CRL verification has made some headway in Erlang (see
> the ssl module docs and https://github.com/Vagabond/erl_crl_example
> <http://erlang.org/doc/man/ssl.html>). You’ll also want to look at
> hostname verification (
> https://github.com/deadtrickster/ssl_verify_hostname.erl
> <https://github.com/benoitc/ssl_verify_hostname>).
>
> HTTPS is as secure as you make it.
>
> Felix mentioned ESL’s lhttpc repo. I’d take a look at Talko’s:
> https://github.com/talko/lhttpc It’s the underlying HTTP client for
> erlcloud.
>
> Hackney also has nice multipart/form functionality.
>
> Cheers,
> Drew
>
> On Dec 12, 2014, at 3:45 PM, Mark Nijhof <mark.nijhof@REDACTED>
> wrote:
>
> I meant to mention Hackney. Why would you choose Hackney over Gun and vise
> verse?
>
> On Fri, Dec 12, 2014 at 9:27 PM, Alex Shneyderman <a.shneyderman@REDACTED
> > wrote:
>
>> nobody mentioned hackney: well written, mature, filled with features. not
>> to mention very active.
>>
>> On Fri, Dec 12, 2014 at 3:17 PM, Mark Nijhof <
>> mark.nijhof@REDACTED> wrote:
>>
>>> Hi Felix, Iñaki,
>>>
>>> Thanks for your reply, you confirm what I was thinking myself about
>>> using Gun but I asked because of:
>>>
>>> > In general the state of http clients in erlang is a bewildering,
>>> overgrown thicket of ancient decaying masonry and beguiling dead ends
>>> filled with poisonous invisible gila monsters.
>>>
>>> Cheers,
>>>
>>> -Mark
>>>
>>>
>>> On Fri, Dec 12, 2014 at 9:03 PM, Felix Gallo <felixgallo@REDACTED>
>>> wrote:
>>>
>>>> httpc has some weird bugs under load and should be retired from the
>>>> standard distribution.
>>>>
>>>> lhttpc is better but is a dead project and has a variety of forks, some
>>>> of which are buggy or incomplete.  The 'esl' fork seems to be the closest
>>>> although in my experience it seems to have a broken pooling mechanism.
>>>> Additionally it uses 'let it crash' for the common case of timeouts, which
>>>> can fill up crash.log quickly and impede investigation into real issues.
>>>>
>>>> dlhttpc is a fork of lhttpc that ferd put together to handle high
>>>> volume requests to a low number of endpoints.  It's also not actively
>>>> maintained, but ferd is still alive and kicking and has recently responded
>>>> to pull requests there.  I intended on using this but rustled up my own
>>>> nasty pool mechanism on top of my own hacked fork of lhttpc.
>>>>
>>>> gun appears to be the most actively maintained; I haven't tried it yet
>>>> but if it's as solid as cowboy, this is probably the right one to use for
>>>> new projects.
>>>>
>>>> shotgun is just an SSE convenience wrapper on gun, so if you don't need
>>>> SSE, sticking with gun is probably your best bet.
>>>>
>>>> fusco is an alpha quality http client that doesn't appear to be
>>>> actively maintained.
>>>>
>>>> In general the state of http clients in erlang is a bewildering,
>>>> overgrown thicket of ancient decaying masonry and beguiling dead ends
>>>> filled with poisonous invisible gila monsters.
>>>>
>>>> F.
>>>>
>>>>
>>>> On Fri, Dec 12, 2014 at 11:44 AM, Iñaki Garay <igarai@REDACTED> wrote:
>>>>>
>>>>> We love our tool shotgun[1], built on top of gun:
>>>>>
>>>>> [1] https://github.com/inaka/shotgun
>>>>>
>>>>> It improves on gun's SSE support.
>>>>>
>>>>> good luck,
>>>>> Iñaki
>>>>>
>>>>> On Fri, Dec 12, 2014 at 4:38 PM, Mark Nijhof <
>>>>> mark.nijhof@REDACTED> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am looking for the atm best solution to make http requests, from
>>>>>> downloading a small file till larger (100mb) archives. I know of f.ex. the
>>>>>> httpc, ibrowse, gun.
>>>>>>
>>>>>> Any preferences? And why?
>>>>>>
>>>>>> -Mark
>>>>>>
>>>>>> --
>>>>>> Mark Nijhof
>>>>>> t:   @MarkNijhof <https://twitter.com/MarkNijhof>
>>>>>> s:  marknijhof
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> erlang-questions mailing list
>>>>>> erlang-questions@REDACTED
>>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> erlang-questions mailing list
>>>>> erlang-questions@REDACTED
>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>
>>>>>
>>>
>>>
>>> --
>>> Mark Nijhof
>>> t:   @MarkNijhof <https://twitter.com/MarkNijhof>
>>> s:  marknijhof
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>
>
> --
> Mark Nijhof
> t:   @MarkNijhof <https://twitter.com/MarkNijhof>
> s:  marknijhof
>
>  _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
>


-- 
Mark Nijhof
t:   @MarkNijhof <https://twitter.com/MarkNijhof>
s:  marknijhof
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20141212/551a3414/attachment.htm>