badmatch with chunked data (LONG)

Michael McDaniel <>
Fri Nov 18 17:52:34 CET 2005


On Fri, Nov 11, 2005 at 08:10:09PM -0800, Michael McDaniel wrote:
> 
> I am apparently having a problem that I thought was fixed in an earlier inets
> release.  Perhaps this is different problem?
> 
> That problem is a badmatch error in inets when receiving chunked data.
> At least, I am pretty sure that is what is happening.
> 
> I make a POST to a server and the response comes back chunked.  I have
> never had problem with this application before when return xml information
> was much smaller (presumably always fit in single chunk before).
> 
> I am on Linux using R10B-8 (inets-4.6.1) or R10B-6 (inets-4.5)
> 
> Follows is the code that generates the problem with 
> the proprietary information scrubbed ...
> 
> 
> Body   = "<some> ... <XML> </XML> </some>" ,
> Server = "server.example.com" ,
> UPW    = "some_username:password" ,
> 
>   case (catch http:request(post,
>   {"https://" ++ Server ,
>   [ 
>      {"Date", httpd_util:rfc1123_date()} ,
>      {"Host", Server} ,
>      {"REALM",  "90X"} ,
>      {"API_VERSION", "T:2.9"} ,
>      {"Authorization", "Basic " ++ http_base_64:encode(UPW)} ,
>      {"Accept", "text/xml/html"},
>      {"User-Agent",  "Erlang Client"} 
>   ], "text/xml; charset=utf-8", Body },
>      [{keepalive, false}, {nodelay,true}], [])) of
>        {ok, Result}    -> {ok, Result} ;
>        {error, Reason} -> 
> 
> 	       write Reason to log and do some other stuff
> .
> 
> 
> ***
> Follows is error with proprietary information scrubbed.
> I added line numbers for reference purposes.
> ***
> 
> 
>   1  =ERROR REPORT==== 11-Nov-2005::02:51:06 ===
>   2 ** Generic server <0.22220.71> terminating 
>   3 ** Last message in was {ssl,{sslsocket,5,<0.22221.71>},
>   4                      <<103,  remaining binary information removed ...
>   5                      10>>}
>   6 ** When Server state == {state,{request,
>   7                         #Ref<0.0.63.220568>,
>   8                         <0.20015.36>,
>   9                         0,
>  10                         https,
>  11                         {"some.example.com",443},
>  12                         "/xmlreceiver",
>  13                        [],
>  14                       post,
>  15                         {http_request_h,
>  16                            undefined,
>  17                            "keep-alive",
>  18                            "Mon, 11 Nov 2005 09:51:05 GMT",
>  19                            undefined,
>  20                            undefined,
>  21                            undefined,
>  22                            undefined,
>  23                            undefined,
>  24                            undefined,
>  25                            "text/xml/html",
>  26                            undefined,
>  27                            undefined,
>  28                            undefined,
>  29                            "Basic zecret",
>  30                            undefined,
>  31                            undefined,
>  32                            "some.example.com/xmlreceiver",
>  33                            undefined,
>  34                            undefined,
>  35                            undefined,
>  36                            undefined,
>  37                            undefined,
>  38                            undefined,
>  39                            undefined,
>  40                            undefined,
>  41                            undefined,
>  42                            [],
>  43                            "Erlang Client",
>  44                            undefined,
>  45                            undefined,
>  46                            undefined,
>  47                            "0",
>  48                            undefined,
>  49                            undefined,
>  50                            undefined,
>  51                            undefined,
>  52                            undefined,
>  53                            undefined,
>  54                            [{"api_version","T:2.9"},
>  55                            {"realm","90X"}]},
>  56                            {"text/xml; charset=utf-8",
>  57                            "<some> ... <XML> </XML> </some>"},
>  58                            {http_options,infinity,true,[],false},
>  59                            "https://some.example.com/xmlreceiver"},
>  60                        {tcp_session,
>  61                            {{"some.example.com",443},<0.22220.71>},
>  62                            false,
>  63                            https,
>  64                            {sslsocket,5,<0.22221.71>},
>  65                            1},
>  66                        {"HTTP/1.1",200,"OK"},
>  67                        {http_response_h,
>  68                            undefined,
>  69                            undefined,
>  70                            "Mon, 11 Nov 2005 09:51:05 GMT",
>  71                            undefined,
>  72                            undefined,
>  73                            "chunked",
>  74                            undefined,
>  75                            undefined,
>  76                            undefined,
>  77                            undefined,
>  78                            undefined,
>  79                            undefined,
>  80                            undefined,
>  81                            undefined,
>  82                            undefined,
>  83                            "Sun-ONE-Web-Server/6.1",
>  84                            undefined,
>  85                            undefined,
>  86                            undefined,
>  87                            undefined,
>  88                            undefined,
>  89                            "0",
>  90                            undefined,
>  91                            undefined,
>  92                            undefined,
>  93                            "text/xml",
>  94                            undefined,
>  95                            undefined,
>  96                            []},
>  97                        undefined,
>  98                    {http_chunk,
>  99                        decode_trailer,
> 100                        [<<>>,
> 101                       "nitsil/<> sdrawkcab s'taht ffuts erom <>",
> 102                        [],
> 103                        nolimit,
> 104                        <<60,  remaining binary information removed ...
> 105                        115>>,
> 106                        "16384"]},
> 107                      {[],[]},
> 108                      new,
> 109                      [],
> 110                         nolimit,
> 111                         nolimit,
> 112                       {options,{undefined,[]},0,2,2,disabled,enabled},
> 113                       {timers,[],undefined}}
> 114 ** Reason for termination == 
> 115 ** {{badmatch,{[[],
> 116                "0",
> 117                "><response> ...  <xml> ... </xml> </response>\n"],
> 118               []}},
> 119    [{http_response,headers,2},
> 120     {http_chunk,handle_headers,2},
> 121     {httpc_handler,handle_http_msg,2},
> 122     {gen_server,handle_msg,6},
> 123     {proc_lib,init_p,5}]}
> 
> 
> I hope I left enough information to be useful for someone to assist in
> resolving this problem.
> 
> A very interesting item is on line 101, the data is all reversed prior
> to the binary data.  I think that is an artifact of the error information
> but maybe not.
> 
> I ran the same POST request to the same server using curl, and received
> proper return XML information which takes up more than 8192 octets (hence
> my thought about problem with chunked data with inets).
> 
> Any suggestions are welcome as now I have run out of ideas on how to
> resolve problem.
> 
> thanks,
> 
> ~Michael
> Portland, Oregon, USA
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Also, Chandru's ibrowse works correctly.

I would prefer to use the inets library directly rather than extra code.


~Michael



More information about the erlang-questions mailing list