<div dir="ltr"><div><div><br></div>It is intended this behavior and only the headers are read<br></div><div>and only a part of them because neither query params of a GET<br></div><div>are not read fro socket.<br><br></div><div>So, basically the body is not read in a dispatchmod, right? <br></div><div>Only a part of the headers. Or all?<br></div><div><br></div><div>And then in dispatchmod how do I know there are unparsed headers<br></div><div>or not to know exactly when to parse the body of the request ?<br><br></div><div>Thanks,<br></div><div>Bogdan<br></div><div><br></div><br><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jun 30, 2015 at 3:36 PM, Steve Vinoski <span dir="ltr"><<a href="mailto:vinoski@ieee.org" target="_blank">vinoski@ieee.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Tue, Jun 30, 2015 at 6:38 AM, Bogdan Andu <span dir="ltr"><<a href="mailto:bog495@gmail.com" target="_blank">bog495@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div>I know intercept module does not have clidata populated.<br><br></div>I was saying that in dispatch module I want POST data.<br><br></div>In a configuration like this:<span><br><br>....<br><server localhost><br> port = 8088<br> listen = 127.0.0.1<br> listen_backlog = 100<br> dispatchmod = dispatch_rewrite<br> docroot = /tmp<br> revproxy = / <a href="http://127.0.0.1:8080/" target="_blank">http://127.0.0.1:8080/</a> intercept_mod intercept_cgi<br></server><br><br></span></div>I thought dispatch_rewrite to give me clidata, but clidata for a POST <br>to a cgi script remains undefined.<br></div></div></div></blockquote><div><br></div></span><div>As its name implies, a dispatch module is useful when you're taking over dispatching from Yaws. For example, I've used it for a video delivery application and for integrating webmachine and Yaws, the latter because webmachine already performs all its own request processing and reply delivery. For the dispatchmod case Yaws reads just enough of the request to build a minimal #arg{} and assumes the dispatch module will handle the rest, including reading additional data from the socket when warranted, so if your dispatchmod is expecting POSTs, it will have to handle them itself.</div><div><div class="h5"><div><br></div><div>--steve</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><br></div>Thanks,<br></div>Bogdan<br><div><div><br></div></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 29, 2015 at 8:48 PM, Steve Vinoski <span dir="ltr"><<a href="mailto:vinoski@ieee.org" target="_blank">vinoski@ieee.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Mon, Jun 29, 2015 at 11:10 AM, Bogdan Andu <span dir="ltr"><<a href="mailto:bog495@gmail.com" target="_blank">bog495@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div>Thanks for reply.<br><br></div>I tested myself yaws in revproxy and <br></div>I like it.<br><br></div><div>Although I don't know how to capture POST data<br></div><div>which should be present in Arg#arg.clidata field <br></div><div>which is also undefined.<br></div><div><br></div><div>I searched the web and docs and found nothing.<br></div><div><br></div><div>I wrongly assumed that field Arg#arg.querydata hold POST data<br></div><br></div>I want POST data to apply some regular expression checks <br>(like mod_rewrite in Apache) on them <br></div></div></div></blockquote><div><br></div></span><div>Pretty sure an intercept_mod has access only to information about the request, and to the headers. The revproxy code uses an internal state record to track details about POST data and such, but an intercept mod doesn't have access to that state. You might consider posting an issue to the yaws github project (<a href="https://github.com/klacke/yaws" target="_blank">https://github.com/klacke/yaws</a>) to see if this functionality can be added.</div><span><font color="#888888"><div><br></div><div>--steve</div></font></span><div><div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><br></div>Thanks,<br></div>Bogdan<br><br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 29, 2015 at 5:49 PM, Steve Vinoski <span dir="ltr"><<a href="mailto:vinoski@ieee.org" target="_blank">vinoski@ieee.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Sun, Jun 28, 2015 at 11:20 AM, Bogdan Andu <span dir="ltr"><<a href="mailto:bog495@gmail.com" target="_blank">bog495@gmail.com</a>></span> wrote:<br></span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div><div><div><div><div><div><div>Hi,<br><br></div>I know the docs<br></div><span>and I run internally Yaws in reverse proxy mode<br></span></div><span><div>and I want this in Internet facing setup also<br></div><div><br></div>I searched the net 'yaws reverse proxy' and I found:<br><br>1) <a href="http://stackoverflow.com/questions/917546/has-anybody-used-yaws-server-as-an-http-proxy" target="_blank">http://stackoverflow.com/questions/917546/has-anybody-used-yaws-server-as-an-http-proxy</a><br></span></div><span> although 6 years old, is Yaws in reverse proxy mode comparable or even<br></span></div><span> better than varnis, haproxy, nginx?<br></span></div></div></div></div></div></div></div></div></div></div></blockquote><div><br></div><div>That question/answer is out of date, since revproxy code was largely refactored in the 2012-2014 timeframe.</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div><div>2) <a href="http://osdir.com/ml/web.server.yaws.general/2007-12/msg00000.html" target="_blank">http://osdir.com/ml/web.server.yaws.general/2007-12/msg00000.html</a><br></div> this one is from klacke and he speaks about difficulties he<br></div> encountered writing revproxy <a href="http://engine.as" target="_blank">engine.as</a> well as an OTP limitation<br></div> being the main obstacle in overcoming these. <br></div> Are these still apply today?<br></div></div></div></div></div></div></blockquote><div><br></div></span><div>Probably not, given the rewrite.</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div>3) also found this: <a href="http://www.erlang-factory.com/upload/presentations/752/reed-efsf2013-whatsapp.pdf" target="_blank">http://www.erlang-factory.com/upload/presentations/752/reed-efsf2013-whatsapp.pdf</a><br></div></div> it seems whatsapp useses or used yaws in revproxy mode with some tweaks<br><br></div>The setup I want is simple:<br><br></div><div>Because I have the applications written in other language<br></div><div>and for rewriting them in Erlang I dont have the time I must <br></div><div>use this setup:<br></div><div><br></div> - Yaws in front-end in revproxy mode with interception module with plenty of check<br></div>on headers, cookies, etc<br><div><div><div><div><div><div><div><div> - Twiggy/Starlet/Starman as back-end or psgi server<br></div><div> more info here: <a href="http://www.slideshare.net/kazeburo/yapc2013psgi-plack" target="_blank">http://www.slideshare.net/kazeburo/yapc2013psgi-plack</a><br><br></div><div> I want to keepalive connections between yaws and psgi servers to avoid<br></div><div> 3-way handshake overhead. Although with outside world I want disable keepalive<br></div><div> and set 'connection: close' and all of thiese can be done by altering the headers<br></div><div> tru interception module.<br><br></div><br><div>Is it safe to keepalive connections between Yaws and Twiggy for<br></div><div>example which is an Libevent implementation web server? <br>Or Starlet/Starman like web server which has a parallel pre-fork model?<br></div></div></div></div></div></div></div></div></div></blockquote><div><br></div></span><div>Seems like it should be OK but I've never used the backend servers you're talking about, so I can't say for sure whether it's safe or not. You might try asking on the erlyaws mailing list (see <a href="https://lists.sourceforge.net/lists/listinfo/erlyaws-list" target="_blank">https://lists.sourceforge.net/lists/listinfo/erlyaws-list</a>).</div><span><font color="#888888"><div><br></div><div>--steve</div></font></span><div><div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div></div><div> <br></div><div>Thanks and sorry for long post,<br><br></div><div>Bogdan<br></div><div><br><div><div><div><div><br><div><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Jun 28, 2015 at 4:51 AM, Steve Vinoski <span dir="ltr"><<a href="mailto:vinoski@ieee.org" target="_blank">vinoski@ieee.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Sat, Jun 27, 2015 at 4:58 AM, Bogdan Andu <span dir="ltr"><<a href="mailto:bog495@gmail.com" target="_blank">bog495@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi,<br><br></div>About yaws as reverse proxy..<br><br></div><div>I want to use yaws as a reverse proxy in a<br></div><div>http -> http setup. no ssl involved whatsoever.<br></div><div><br></div><div>I am interested in interception module where I want <br></div><div>to apply various checks on headers, query string, etc<br></div><div>making this some kind of www firewall .<br></div><div><br>Is this feature of yaws production ready ?<br></div></div></div></div></blockquote><div><br></div></span><div>Yes. You can find details about it in chapter 13 of <a href="http://yaws.hyber.org/yaws.pdf" target="_blank">http://yaws.hyber.org/yaws.pdf</a>, or under the revproxy section of <a href="http://yaws.hyber.org/yman.yaws?page=yaws.conf" target="_blank">http://yaws.hyber.org/yman.yaws?page=yaws.conf</a> .</div><span><font color="#888888"><div><br></div><div>--steve</div></font></span><span><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div></div><br><br></div>Thanks,<br></div>Bogdan<br><br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 13, 2015 at 6:14 PM, Steve Vinoski <span dir="ltr"><<a href="mailto:vinoski@ieee.org" target="_blank">vinoski@ieee.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">I don't recall seeing Yaws users asking for this config feature in the past, so it's unlikely we'll add it. But what you're asking for -- a configuration point for methods -- would be implemented much as I've shown in my previous emails, much like a dispatchmod. The dispatchmod is as early in the request handling process as you can get after the formation of the #arg{}. The dispatchmod code I provided requires less configuration than what you're showing, even for the default case, plus if having to have a new module concerns you, the dispatch/1 function can be added to some other existing module you already have instead.<span><font color="#888888"><div><br></div><div>--steve<br><div><div><br></div><div><br></div></div></div></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 13, 2015 at 4:30 AM, Bogdan Andu <span dir="ltr"><<a href="mailto:bog495@gmail.com" target="_blank">bog495@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div>Yes but the point is to have a default configuration that can be overridden by such a mechanism<br></div>if one is configured.<br></div>The 99 percent of cases only need a default behaviour.<br><br></div>The way I see this is to have something like that (all in one):<br><br></div><LIMIT POST GET><br></div><div> mod_405=my_405_handle_module<br>....<br></div></LIMIT><br><br></div>in this way we can also customize the response if a method other than GET or POST is sent to the server<br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 13, 2015 at 10:17 AM, Imants Cekusins <span dir="ltr"><<a href="mailto:imantc@gmail.com" target="_blank">imantc@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span>> Traffic with methods not allowed should be discarded with 405<br>
<br>
</span>you see, someone else might prefer another action depending on method<br>
not allowed.<br>
<br>
a dedicated attribute may be convenient but then someone would ask:<br>
"how do I change the response code? how do I redirect?". Current<br>
approach gives you choice.<br>
<br>
one of those cases when there is more than one approach, a prefers A,<br>
b prefers B. Both have a valid point.<br>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></span></div><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>