[erlang-questions] Any tcp related changes in R15B* that might cause this?

[Anthony Molinaro]

Hi Steve,

It may not be directly caused by your patch, but that was the first
patch that stuck out when I started looking through the readmes for the
releases.  I've not been able to untangle mochiweb enough to do a test
without it, so attached is a test file which uses mochiweb.

If you compile it, you should see this with R14B04

Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.5  (abort with ^G)
1> mochi_test:start().
{ok,<0.44.0>}
2> mochi_test:test(10000).
{ok,{{"HTTP/1.1",400,"Bad Request"},
     [{"date","Thu, 13 Dec 2012 18:36:06 GMT"},
      {"server","MochiWeb/1.0 (Any of you quaids got a smint?)"},
      {"content-length","0"}],
     []}}
3> 

and this with R15B02

Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:2:2] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.2  (abort with ^G)
1> mochi_test:start().
{ok,<0.44.0>}
2> mochi_test:test(10000).
{error,socket_closed_remotely}
3>

I used Mochiweb 2.3.1 and compiled it and the test code with R14B04, then
ran the test with the 2 different versions.  Discovering the exact errors
consisted of adding some debug output to mochiweb_http:headers/5 and
mochiweb_http:request/2 in the _Other clauses.

After talking with the mochiweb devs on the mochiweb group, I'll probably
submit a patch there to deal with the tcp_error, but it would be interesting
to know what caused the behavior change.

Looking forward to any results you might get,

-Anthony


On Wed, Dec 12, 2012 at 08:35:11PM -0500, Steve Vinoski wrote:
> On Wed, Dec 12, 2012 at 7:24 PM, Anthony Molinaro <
> anthonym@REDACTED> wrote:
> 
> > Okay, I found it.  Looks like this is the cause
> >
> > OTP-9389  Honor option packet_size for http packet parsing by both TCP
> >           socket and erlang:decode_packet. This gives the ability to
> >           accept HTTP headers larger than the default setting, but also
> >           avoid DoS attacks by accepting lines only up to
> >           whatever length you wish to allow. For consistency,
> >           packet type line also honor option packet_size. (Thanks
> >           to Steve Vinoski)
> >
> > With R14B04 when you have a header which is too long, your process
> > gets a message like
> >
> >
> > {http,#Port<0.120387186>,{http_error,"X2qutbTBbXg7VHgDDhGrEvDbzuxiyDlI7VFloMyAJKVqY2fTEkMc70UchLPRG8Cjowzmib4KszbCRwA5IBbAd2MbRi5X_tK2nfRtheavXdhQv8XbinzmhCM1E9YCeuFAg_9TfqUS0sWUd52mgjkWGqNe4Z9S0IxFYnFtf5..."}}
> >
> > in R15B02 you get a message like
> >
> > {tcp_error,#Port<0.104208233>,emsgsize}
> >
> > The prior case is handled correctly by mochiweb, the latter is not.
> >
> 
> 
> I just ran a simple test case against the commit prior to 5984409d and then
> against 5984409d, in both cases sending an HTTP header of various lengths,
> and in both cases always received http_error, and never tcp_error. The
> server simply did a receive with the socket in http_bin mode.
> 
> How long is the header in your case? Do you have a test case you can send
> me?
> 
> 
> I'm not sure if the commits for OTP-9389 actually caused this change or if
> > it was some other change.  Also, not sure if it was meant to be a backward
> > compatible change or not (the comment on the commit
> >
> > https://github.com/erlang/otp/commit/5984409d1264871cbe61bfec875de53e51713efb
> > seems to suggest it was supposed to be backward compatible, but maybe this
> > was a side effect?  It seems like emsgsize is more correct.
> >
> 
> 
> I think the intent was to be backward compatible; I seem to remember that
> putting the patch together was very difficult and time-consuming due to
> trying to achieve that, with numerous iterations and reviews.
> 
> --steve

> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions


-- 
------------------------------------------------------------------------
Anthony Molinaro                           <anthonym@REDACTED>
-------------- next part --------------
-module(mochi_test).

-export([start/0,
         stop/0,
         handle_http/1,
         test/1]).

start() ->
  application:start (inets),
  mochiweb_http:start([{port, 5678}, {loop, fun(Req) -> handle_http(Req) end}]).

stop() ->
  mochiweb_http:stop().

handle_http(Req) ->
  Req:respond({ 200,
                [ {"Content-Type", "text/html"} ],
                [ "<html><body>Hello</body></html>" ]
              }).

test (Len) ->
  Random =
    get_random_string (Len,
      "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"),
  httpc:request (get, {"http://127.0.0.1:5678/",
                 [{"X-Random", Random}]}, [], []).
 

% grabbed from
%   http://blog.teemu.im/2009/11/07/generating-random-strings-in-erlang/
get_random_string(Length, AllowedChars) ->
  lists:foldl(fun(_, Acc) ->
                [lists:nth(random:uniform(length(AllowedChars)),
                 AllowedChars)] ++ Acc
              end, [], lists:seq(1, Length)).