<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">Le 20 avr. 2015 à 13:05, John Duffy <<a href="mailto:jb_duffy@btinternet.com" class="">jb_duffy@btinternet.com</a>> a écrit :</div><br class="Apple-interchange-newline"><div class="">Hi Jesper<div class=""><br class=""></div><div class="">Thank you for your reply, very helpful.</div><div class=""><br class=""></div><div class="">I'm still a bit puzzled as to why my test doesn't work, I'm being to wonder if I need to be sending additional headers, or something, to the server.</div><div class=""><br class=""></div><div class="">If I use 'curl' then everything works, I get a steady stream of data...</div><div class=""><br class=""></div><div class="">curl "<a href="http://stream-sandbox.oanda.com/v1/prices?accountId=99999&instruments=EUR_USD" class="">http://stream-sandbox.oanda.com/v1/prices?accountId=99999&instruments=EUR_USD</a>"</div></div></blockquote><div><br class=""></div><div>The problem seems to be the server because if you use -v for curl you will observe something like this:</div><div><div><br class=""></div><div>< HTTP/1.1 200 Ok</div><div>* Server openresty/1.7.0.1 is not blacklisted</div><div>< Server: openresty/1.7.0.1</div><div>< Date: Tue, 21 Apr 2015 15:15:16 GMT</div><div>< Content-Type: application/json</div><div>< Transfer-Encoding: chunked</div><div>< Connection: close</div><div>< Access-Control-Allow-Origin: *</div><div><</div><div>{"tick":{"instrument":"EUR_CHF","time":"2015-04-21T15:13:48.585046Z","bid":1.2041,"ask":1.20435}}</div><div>{"heartbeat":{"time":"2015-04-21T15:15:16.632703Z"}}</div><div>{"heartbeat":{"time":"2015-04-21T15:15:21.632772Z"}}</div><div>{"heartbeat":{"time":"2015-04-21T15:15:24.598956Z"}}</div><div><br class=""></div><div>As you can see there's some strange headers:</div><div>Connection: close </div><div>The connection is not closed because it's a stream.</div><div><br class=""></div><div>Transfer-Encoding: chunked</div><div>What I see is absolutely not chunked transfer, it's a bunch of json lines...</div><div>Chunks should be preceded by the size (hex encoded) and \r\n</div><div><br class=""></div><div>So I think that the erlang code is trying to respect headers, close the connection and search for chunk encoding, but there's none...</div><div><br class=""></div><div>It might be interesting to report those problems to the developers of this service :)</div><div><br class=""></div><div><br class=""></div><div><br class=""></div></div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class=""></div><div class="">However, putting the same URL into your example results in a single time-out and the Erlang emulator stalling.</div><div class=""><br class=""></div><div class="">Kind regards</div><div class=""><br class=""></div><div class="">John</div><div class=""><br class=""></div><div class=""><blockquote style="margin-right: 0px; margin-left:15px;" class="">----Original message----<br class="">From : <a href="mailto:jesper.louis.andersen@gmail.com" class="">jesper.louis.andersen@gmail.com</a><br class="">Date : 19/04/2015 - 22:00 (GMTDT)<br class="">To : <a href="mailto:jb_duffy@btinternet.com" class="">jb_duffy@btinternet.com</a>, <a href="mailto:erlang-questions@erlang.org" class="">erlang-questions@erlang.org</a><br class="">Subject : Re: [erlang-questions] Streaming Data using httpc<br class=""><br class=""><div dir="ltr" class=""><br class=""><br class=""><div class="gmail_quote">On Sun, Apr 19, 2015 at 6:14 PM John Duffy <<a href="mailto:jb_duffy@btinternet.com" class="">jb_duffy@btinternet.com</a>> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br class=""><br class="">-module(streaming).<br class=""><br class=""></blockquote><div class=""><br class=""></div><div class="">[...]</div><div class=""><br class=""></div><div class="">You are pretty close to the goal, but you are confusing the stream/receiver options I think. In streaming, you will receive the data as a series of chunks, which is what your code expect, but you don't supply an option requesting streaming operation. So you don't retrieve an expected tuple. You can add a catchall tuple to your receieve clause in receive_data/1 to make sure you have the right format in your match. Also, you can add a 'after' clause to time out after a while. This can make debugging easier since you "get back" to the the REPL.</div><div class=""><br class=""></div><div class="">The following works on a quick test in my end. Note how the receive clause is different from yours, and that you get everything in one fell swoop, rather than having to match on a multitude of clauses.</div><div class=""><br class=""></div><div class="">For more serious work, you might want to check out some of the numerous other projects for HTTP client requests. I'm partial to Gun and Hackney myself, but there are also ibrowse, lhttpc and fusco. They have slightly different semantics and areas at which they excel, so choose wisely :)</div><div class=""><br class=""></div><div class="">For a prototype however, I think httpc is fine.</div><div class=""><br class=""></div><div class="">-module(streaming).</div><div class=""><br class=""></div><div class="">-export([data/0]). </div><div class=""><br class=""></div><div class="">data() -></div><div class=""> {ok, RequestId} = httpc:request(get, {"<a href="http://example.com/" class="">example.com</a>", []}, [], [{sync, false}, {receiver, self()}]),</div><div class=""> receive_data(RequestId).</div><div class=""><br class=""></div><div class="">receive_data(RequestId) -></div><div class=""> receive</div><div class=""> {http, {RequestId, {StatusLine, Headers, Body}}} -></div><div class=""> error_logger:info_report(</div><div class=""> <span class="Apple-tab-span" style="white-space:pre"> </span>#{</div><div class=""> <span class="Apple-tab-span" style="white-space:pre"> </span> status => StatusLine,</div><div class=""> <span class="Apple-tab-span" style="white-space:pre"> </span> headers => length(Headers),</div><div class=""> <span class="Apple-tab-span" style="white-space:pre"> </span> body_size => byte_size(Body)</div><div class=""> <span class="Apple-tab-span" style="white-space:pre"> </span> }),</div><div class=""> ok</div><div class=""> after 5000 -></div><div class=""> error_logger:info_report(timeout)</div><div class=""> end.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br class="">
</blockquote></div></div>
<br class=""></blockquote><br class=""><div class=""><br class="webkit-block-placeholder"></div></div>_______________________________________________<br class="">erlang-questions mailing list<br class=""><a href="mailto:erlang-questions@erlang.org" class="">erlang-questions@erlang.org</a><br class="">http://erlang.org/mailman/listinfo/erlang-questions<br class=""></div></blockquote></div><br class=""></body></html>