httpc:request pipelining example and blocking problem

[Anthony Howe]

On 2021-12-17 20:19, Yao Bao wrote:
> Hello,
> 
> It is good to see users of inets still exists even though for exercises. And
> luckily, it still exists within Erlang/OTP by default.

Well partially an exercise to remain light weight and keep it basic because that
should be all that is needed, but it is also a work thing.

> Tried inets httpd to build a simple http server, it works fine. Someday I
> might finally realize some advanced features of a modern http server
> provides is essential to my application, but I prefer starting from basic/poor
> things.

Well this particular task I need both httpd and httpc.  The small REST server
part worked fine, despite the poor documentation.  Took me a while to realise
you need to add your own modules to the list, but once past that it was fine.

> For your question:
> 1. Haven't use inets httpc, so no advice personally.
> 2. If it worked with Ruby based service but not Erlang 19 based Cowboy based
> service, the problem might be the communication. If the http request and response
> works fine, and the process just blocks, I would guess the problem comes from the
> "driver" of the inets httpc, it receives the http message, and turn it into an Erlang
> message, might be the Erlang message which been putted into process's mailbox
> does not match any of the designed pattern, so the "receive" expression blocks and
> timed out. Or, the http driver does not send message back to the client process for
> some reason.

Was working on this more last night.

* One part of the issue was the server was sending a HTTP chunked response and I
needed to switch to using `stream`.

* The other issue is I'm using a `gen_server` to drive part of the client aspect
and I needed to restructure how I setup the receive loop.  Now I get the 1st
chunk (more often), but still appears to block on occasion on the chunk.

> More information about your situation would be helpful for us to find the problem:
> - which version of inets httpc (and version of Erlang/OTP) do you use?

This testing tool is using Erlang 21 (prebuilt Ubuntu 20.4 package).

> - which version of Cowboy (you mentioned Erlang 19) do you use?

The backend is using Erlang 19 and a similarly old version of Cowboy.  Not sure
of the version.  The backend I don't have much influence over (yet), but it
works in production and the client doesn't want to fiddle.


> The source code of inets httpc is not huge, if you want to know where exactly the
> message is lost, give it a try to dig the source code.

I was hoping to avoid that, but alas.

Anthony

> 
> Cheers,
> Yao
> 
>>
>> I'm trying to exclusively use inets httpc for some simple REST request.  Please
>> do NOT suggest other Erlang HTTP clients; the exercise is to only use inets
>> httpc.  The docs for the httpc client are pretty poor and online searches
>> haven't helped.
>>
>> Two questions:
>>
>> 1. Does anyone have an example of using inets httpc to do pipeline requests?
>>
>> 2. I've managed a single asynchronous request to a Ruby based service which
>> worked fine.  Now when I try make a request to an Erlang Cowboy based service
>> (an old Erlang 19 3rd party application), the request goes through and the reply
>> sent (as observed using tcpdump), but the process blocks until the timeout,
>> never returning the response.
>>
>> -- 
>> Anthony C Howe         	                                       SnertSoft
>> achowe@REDACTED     Twitter: SirWumpus            BarricadeMX & Milters
>> http://snert.com/    http://nanozen.snert.com/     http://snertsoft.com/
> 
> 


-- 
Anthony C Howe         	                                       SnertSoft
achowe@REDACTED     Twitter: SirWumpus            BarricadeMX & Milters
http://snert.com/    http://nanozen.snert.com/     http://snertsoft.com/