badmatch with chunked data (LONG)

[Michael McDaniel]

I am apparently having a problem that I thought was fixed in an earlier inets
release.  Perhaps this is different problem?

That problem is a badmatch error in inets when receiving chunked data.
At least, I am pretty sure that is what is happening.

I make a POST to a server and the response comes back chunked.  I have
never had problem with this application before when return xml information
was much smaller (presumably always fit in single chunk before).

I am on Linux using R10B-8 (inets-4.6.1) or R10B-6 (inets-4.5)

Follows is the code that generates the problem with 
the proprietary information scrubbed ...


Body   = "<some> ... <XML> </XML> </some>" ,
Server = "server.example.com" ,
UPW    = "some_username:password" ,

  case (catch http:request(post,
  {"https://" ++ Server ,
  [ 
     {"Date", httpd_util:rfc1123_date()} ,
     {"Host", Server} ,
     {"REALM",  "90X"} ,
     {"API_VERSION", "T:2.9"} ,
     {"Authorization", "Basic " ++ http_base_64:encode(UPW)} ,
     {"Accept", "text/xml/html"},
     {"User-Agent",  "Erlang Client"} 
  ], "text/xml; charset=utf-8", Body },
     [{keepalive, false}, {nodelay,true}], [])) of
       {ok, Result}    -> {ok, Result} ;
       {error, Reason} -> 

	       write Reason to log and do some other stuff
.


***
Follows is error with proprietary information scrubbed.
I added line numbers for reference purposes.
***


  1  =ERROR REPORT==== 11-Nov-2005::02:51:06 ===
  2 ** Generic server <0.22220.71> terminating 
  3 ** Last message in was {ssl,{sslsocket,5,<0.22221.71>},
  4                      <<103,  remaining binary information removed ...
  5                      10>>}
  6 ** When Server state == {state,{request,
  7                         #Ref<0.0.63.220568>,
  8                         <0.20015.36>,
  9                         0,
 10                         https,
 11                         {"some.example.com",443},
 12                         "/xmlreceiver",
 13                        [],
 14                       post,
 15                         {http_request_h,
 16                            undefined,
 17                            "keep-alive",
 18                            "Mon, 11 Nov 2005 09:51:05 GMT",
 19                            undefined,
 20                            undefined,
 21                            undefined,
 22                            undefined,
 23                            undefined,
 24                            undefined,
 25                            "text/xml/html",
 26                            undefined,
 27                            undefined,
 28                            undefined,
 29                            "Basic zecret",
 30                            undefined,
 31                            undefined,
 32                            "some.example.com/xmlreceiver",
 33                            undefined,
 34                            undefined,
 35                            undefined,
 36                            undefined,
 37                            undefined,
 38                            undefined,
 39                            undefined,
 40                            undefined,
 41                            undefined,
 42                            [],
 43                            "Erlang Client",
 44                            undefined,
 45                            undefined,
 46                            undefined,
 47                            "0",
 48                            undefined,
 49                            undefined,
 50                            undefined,
 51                            undefined,
 52                            undefined,
 53                            undefined,
 54                            [{"api_version","T:2.9"},
 55                            {"realm","90X"}]},
 56                            {"text/xml; charset=utf-8",
 57                            "<some> ... <XML> </XML> </some>"},
 58                            {http_options,infinity,true,[],false},
 59                            "https://some.example.com/xmlreceiver"},
 60                        {tcp_session,
 61                            {{"some.example.com",443},<0.22220.71>},
 62                            false,
 63                            https,
 64                            {sslsocket,5,<0.22221.71>},
 65                            1},
 66                        {"HTTP/1.1",200,"OK"},
 67                        {http_response_h,
 68                            undefined,
 69                            undefined,
 70                            "Mon, 11 Nov 2005 09:51:05 GMT",
 71                            undefined,
 72                            undefined,
 73                            "chunked",
 74                            undefined,
 75                            undefined,
 76                            undefined,
 77                            undefined,
 78                            undefined,
 79                            undefined,
 80                            undefined,
 81                            undefined,
 82                            undefined,
 83                            "Sun-ONE-Web-Server/6.1",
 84                            undefined,
 85                            undefined,
 86                            undefined,
 87                            undefined,
 88                            undefined,
 89                            "0",
 90                            undefined,
 91                            undefined,
 92                            undefined,
 93                            "text/xml",
 94                            undefined,
 95                            undefined,
 96                            []},
 97                        undefined,
 98                    {http_chunk,
 99                        decode_trailer,
100                        [<<>>,
101                       "nitsil/<> sdrawkcab s'taht ffuts erom <>",
102                        [],
103                        nolimit,
104                        <<60,  remaining binary information removed ...
105                        115>>,
106                        "16384"]},
107                      {[],[]},
108                      new,
109                      [],
110                         nolimit,
111                         nolimit,
112                       {options,{undefined,[]},0,2,2,disabled,enabled},
113                       {timers,[],undefined}}
114 ** Reason for termination == 
115 ** {{badmatch,{[[],
116                "0",
117                "><response> ...  <xml> ... </xml> </response>\n"],
118               []}},
119    [{http_response,headers,2},
120     {http_chunk,handle_headers,2},
121     {httpc_handler,handle_http_msg,2},
122     {gen_server,handle_msg,6},
123     {proc_lib,init_p,5}]}


I hope I left enough information to be useful for someone to assist in
resolving this problem.

A very interesting item is on line 101, the data is all reversed prior
to the binary data.  I think that is an artifact of the error information
but maybe not.

I ran the same POST request to the same server using curl, and received
proper return XML information which takes up more than 8192 octets (hence
my thought about problem with chunked data with inets).

Any suggestions are welcome as now I have run out of ideas on how to
resolve problem.

thanks,

~Michael
Portland, Oregon, USA