[erlang-questions] configure http methods in yaws
Bogdan Andu
bog495@REDACTED
Tue Jun 30 17:09:53 CEST 2015
It is intended this behavior and only the headers are read
and only a part of them because neither query params of a GET
are not read fro socket.
So, basically the body is not read in a dispatchmod, right?
Only a part of the headers. Or all?
And then in dispatchmod how do I know there are unparsed headers
or not to know exactly when to parse the body of the request ?
Thanks,
Bogdan
On Tue, Jun 30, 2015 at 3:36 PM, Steve Vinoski <vinoski@REDACTED> wrote:
>
>
> On Tue, Jun 30, 2015 at 6:38 AM, Bogdan Andu <bog495@REDACTED> wrote:
>
>> I know intercept module does not have clidata populated.
>>
>> I was saying that in dispatch module I want POST data.
>>
>> In a configuration like this:
>>
>> ....
>> <server localhost>
>> port = 8088
>> listen = 127.0.0.1
>> listen_backlog = 100
>> dispatchmod = dispatch_rewrite
>> docroot = /tmp
>> revproxy = / http://127.0.0.1:8080/ intercept_mod intercept_cgi
>> </server>
>>
>> I thought dispatch_rewrite to give me clidata, but clidata for a POST
>> to a cgi script remains undefined.
>>
>
> As its name implies, a dispatch module is useful when you're taking over
> dispatching from Yaws. For example, I've used it for a video delivery
> application and for integrating webmachine and Yaws, the latter because
> webmachine already performs all its own request processing and reply
> delivery. For the dispatchmod case Yaws reads just enough of the request to
> build a minimal #arg{} and assumes the dispatch module will handle the
> rest, including reading additional data from the socket when warranted, so
> if your dispatchmod is expecting POSTs, it will have to handle them itself.
>
> --steve
>
>
>
>>
>> Thanks,
>> Bogdan
>>
>>
>> On Mon, Jun 29, 2015 at 8:48 PM, Steve Vinoski <vinoski@REDACTED> wrote:
>>
>>>
>>>
>>> On Mon, Jun 29, 2015 at 11:10 AM, Bogdan Andu <bog495@REDACTED> wrote:
>>>
>>>> Thanks for reply.
>>>>
>>>> I tested myself yaws in revproxy and
>>>> I like it.
>>>>
>>>> Although I don't know how to capture POST data
>>>> which should be present in Arg#arg.clidata field
>>>> which is also undefined.
>>>>
>>>> I searched the web and docs and found nothing.
>>>>
>>>> I wrongly assumed that field Arg#arg.querydata hold POST data
>>>>
>>>> I want POST data to apply some regular expression checks
>>>> (like mod_rewrite in Apache) on them
>>>>
>>>
>>> Pretty sure an intercept_mod has access only to information about the
>>> request, and to the headers. The revproxy code uses an internal state
>>> record to track details about POST data and such, but an intercept mod
>>> doesn't have access to that state. You might consider posting an issue to
>>> the yaws github project (https://github.com/klacke/yaws) to see if this
>>> functionality can be added.
>>>
>>> --steve
>>>
>>>
>>>
>>>> Thanks,
>>>> Bogdan
>>>>
>>>>
>>>> On Mon, Jun 29, 2015 at 5:49 PM, Steve Vinoski <vinoski@REDACTED>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Sun, Jun 28, 2015 at 11:20 AM, Bogdan Andu <bog495@REDACTED>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I know the docs
>>>>>> and I run internally Yaws in reverse proxy mode
>>>>>> and I want this in Internet facing setup also
>>>>>>
>>>>>> I searched the net 'yaws reverse proxy' and I found:
>>>>>>
>>>>>> 1)
>>>>>> http://stackoverflow.com/questions/917546/has-anybody-used-yaws-server-as-an-http-proxy
>>>>>> although 6 years old, is Yaws in reverse proxy mode comparable or
>>>>>> even
>>>>>> better than varnis, haproxy, nginx?
>>>>>>
>>>>>
>>>>> That question/answer is out of date, since revproxy code was largely
>>>>> refactored in the 2012-2014 timeframe.
>>>>>
>>>>>
>>>>>> 2) http://osdir.com/ml/web.server.yaws.general/2007-12/msg00000.html
>>>>>> this one is from klacke and he speaks about difficulties he
>>>>>> encountered writing revproxy engine.as well as an OTP limitation
>>>>>> being the main obstacle in overcoming these.
>>>>>> Are these still apply today?
>>>>>>
>>>>>
>>>>> Probably not, given the rewrite.
>>>>>
>>>>>
>>>>>> 3) also found this:
>>>>>> http://www.erlang-factory.com/upload/presentations/752/reed-efsf2013-whatsapp.pdf
>>>>>> it seems whatsapp useses or used yaws in revproxy mode with some
>>>>>> tweaks
>>>>>>
>>>>>> The setup I want is simple:
>>>>>>
>>>>>> Because I have the applications written in other language
>>>>>> and for rewriting them in Erlang I dont have the time I must
>>>>>> use this setup:
>>>>>>
>>>>>> - Yaws in front-end in revproxy mode with interception module with
>>>>>> plenty of check
>>>>>> on headers, cookies, etc
>>>>>> - Twiggy/Starlet/Starman as back-end or psgi server
>>>>>> more info here:
>>>>>> http://www.slideshare.net/kazeburo/yapc2013psgi-plack
>>>>>>
>>>>>> I want to keepalive connections between yaws and psgi servers to
>>>>>> avoid
>>>>>> 3-way handshake overhead. Although with outside world I want disable
>>>>>> keepalive
>>>>>> and set 'connection: close' and all of thiese can be done by
>>>>>> altering the headers
>>>>>> tru interception module.
>>>>>>
>>>>>>
>>>>>> Is it safe to keepalive connections between Yaws and Twiggy for
>>>>>> example which is an Libevent implementation web server?
>>>>>> Or Starlet/Starman like web server which has a parallel pre-fork
>>>>>> model?
>>>>>>
>>>>>
>>>>> Seems like it should be OK but I've never used the backend servers
>>>>> you're talking about, so I can't say for sure whether it's safe or not. You
>>>>> might try asking on the erlyaws mailing list (see
>>>>> https://lists.sourceforge.net/lists/listinfo/erlyaws-list).
>>>>>
>>>>> --steve
>>>>>
>>>>>
>>>>>
>>>>>> Thanks and sorry for long post,
>>>>>>
>>>>>> Bogdan
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Jun 28, 2015 at 4:51 AM, Steve Vinoski <vinoski@REDACTED>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Jun 27, 2015 at 4:58 AM, Bogdan Andu <bog495@REDACTED>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> About yaws as reverse proxy..
>>>>>>>>
>>>>>>>> I want to use yaws as a reverse proxy in a
>>>>>>>> http -> http setup. no ssl involved whatsoever.
>>>>>>>>
>>>>>>>> I am interested in interception module where I want
>>>>>>>> to apply various checks on headers, query string, etc
>>>>>>>> making this some kind of www firewall .
>>>>>>>>
>>>>>>>> Is this feature of yaws production ready ?
>>>>>>>>
>>>>>>>
>>>>>>> Yes. You can find details about it in chapter 13 of
>>>>>>> http://yaws.hyber.org/yaws.pdf, or under the revproxy section of
>>>>>>> http://yaws.hyber.org/yman.yaws?page=yaws.conf .
>>>>>>>
>>>>>>> --steve
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Bogdan
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Feb 13, 2015 at 6:14 PM, Steve Vinoski <vinoski@REDACTED>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I don't recall seeing Yaws users asking for this config feature in
>>>>>>>>> the past, so it's unlikely we'll add it. But what you're asking for -- a
>>>>>>>>> configuration point for methods -- would be implemented much as I've shown
>>>>>>>>> in my previous emails, much like a dispatchmod. The dispatchmod is as early
>>>>>>>>> in the request handling process as you can get after the formation of the
>>>>>>>>> #arg{}. The dispatchmod code I provided requires less configuration than
>>>>>>>>> what you're showing, even for the default case, plus if having to have a
>>>>>>>>> new module concerns you, the dispatch/1 function can be added to some other
>>>>>>>>> existing module you already have instead.
>>>>>>>>>
>>>>>>>>> --steve
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Feb 13, 2015 at 4:30 AM, Bogdan Andu <bog495@REDACTED>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Yes but the point is to have a default configuration that can be
>>>>>>>>>> overridden by such a mechanism
>>>>>>>>>> if one is configured.
>>>>>>>>>> The 99 percent of cases only need a default behaviour.
>>>>>>>>>>
>>>>>>>>>> The way I see this is to have something like that (all in one):
>>>>>>>>>>
>>>>>>>>>> <LIMIT POST GET>
>>>>>>>>>> mod_405=my_405_handle_module
>>>>>>>>>> ....
>>>>>>>>>> </LIMIT>
>>>>>>>>>>
>>>>>>>>>> in this way we can also customize the response if a method other
>>>>>>>>>> than GET or POST is sent to the server
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 13, 2015 at 10:17 AM, Imants Cekusins <
>>>>>>>>>> imantc@REDACTED> wrote:
>>>>>>>>>>
>>>>>>>>>>> > Traffic with methods not allowed should be discarded with 405
>>>>>>>>>>>
>>>>>>>>>>> you see, someone else might prefer another action depending on
>>>>>>>>>>> method
>>>>>>>>>>> not allowed.
>>>>>>>>>>>
>>>>>>>>>>> a dedicated attribute may be convenient but then someone would
>>>>>>>>>>> ask:
>>>>>>>>>>> "how do I change the response code? how do I redirect?". Current
>>>>>>>>>>> approach gives you choice.
>>>>>>>>>>>
>>>>>>>>>>> one of those cases when there is more than one approach, a
>>>>>>>>>>> prefers A,
>>>>>>>>>>> b prefers B. Both have a valid point.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150630/6eee82f0/attachment.htm>
More information about the erlang-questions
mailing list