Hi all,<div><br></div><div>I've got a webmachine/mochiweb server up and running accepting SSL connections to transfer files via a streamed body. There are 2 issues that I'm seeing:</div><div><br></div><div>1. Very often the sockets close themselves 'randomly', approximately 5% of the time. Resulting in errors during the write process. Code is at the end of the email.</div>
<div><br></div><div><div>{error,{error,{badmatch,{error,closed}},</div><div> [{webmachine_request,recv_chunked_body,4},</div><div> {file_srv,stream_data_to_disk,2},</div><div> {file_srv,accept_content,2},</div>
<div> {webmachine_resource,resource_call,3},</div><div> {webmachine_resource,do,3},</div><div> {webmachine_decision_core,resource_call,1},</div><div> {webmachine_decision_core,accept_helper,0},</div>
<div> {webmachine_decision_core,decision,1}]}}</div></div><div><br></div><div><div>{error,{exit,normal,</div><div> [{webmachine_request,read_chunk_length,3},</div><div> {webmachine_request,recv_chunked_body,3},</div>
<div> {file_srv,stream_data_to_disk,2},</div><div> {file_srv,accept_content,2},</div><div> {webmachine_resource,resource_call,3},</div><div> {webmachine_resource,do,3},</div>
<div> {webmachine_decision_core,resource_call,1},</div><div> {webmachine_decision_core,accept_helper,0}]}}</div></div><div><br></div><div>These errors both stem from read from the socket. Now my question is, why are closed socket errors occurring so often? How would I go about trying to mitigate and/or prevent these issues from happening?</div>
<div><br></div><div>2. I'm using the 'chunkify' option on the httpc request which allows a stream of data to the webserver, the issue that I'm seeing is that when the connection is forcibly shutdown (ssl socket read error, issue #1), httpc will sometimes attempt another request of the same type automatically! There doesn't seem to be anything that I can see that controls this behaviour. Is there any way to turn this off? Is there something that I'm doing wrong? It seems more like a bug than anything else as it only happens occasionally.</div>
<div><br></div><div>127.0.0.1 - - [08/Nov/2011:18:14:55 -0500] "HEAD /files/test19.txt HTTP/1.1" 200 chunked "" ""</div><div><div>127.0.0.1 - - [08/Nov/2011:18:14:55 -0500] "PUT /files/test19.txt HTTP/1.1" 500 697 "" ""</div>
<div>127.0.0.1 - - [08/Nov/2011:18:14:55 -0500] "PUT /files/test19.txt HTTP/1.1" 500 697 "" ""</div></div><div><br></div><div>This error report is generated on the duplicate retry, but not when it functions properly:</div>
<div><br></div><div><div>=ERROR REPORT==== 8-Nov-2011::18:29:47 ===</div><div>Received unexpected ssl data on {sslsocket,new_ssl,<0.850.0>}</div><div> Data: <<"HTTP/1.1 500 Internal Server Error\r\nServer: MochiWeb/1.1 WebMachine/1.9.0 (someone had painted it blue)\r\nDate: Tue, 08 Nov 2011 23:29:47 GMT\r\nContent-Type: text/html\r\nContent-Length: 697\r\n\r\n">></div>
<div> MFA: undefined</div><div> Request: undefined</div><div> Session: {session,{{"localhost",443},<0.849.0>},</div><div> false,https,</div><div> {sslsocket,new_ssl,<0.850.0>},</div>
<div> {essl,[]},</div><div> 1,keep_alive}</div><div> Status: keep_alive</div><div> StatusLine: undefined</div><div> Profile: httpc_manager</div></div><div><br></div>
<div><br></div><div>Thanks for reading! Any tips would be appreciated.</div><div><br></div><div>Cheers,</div><div>Eric Boyer</div><div><br></div><div><br></div><div><br></div><div>==CODE==</div><div>Server (Webmachine):</div>
<div><br></div><div><div>content_types_accepted(ReqData, Context) -></div><div> CT = case wrq:get_req_header("content-type", ReqData) of</div><div> undefined -> "application/octet-stream";</div>
<div> X -> X</div><div> end,</div><div> {MT, _Params} = webmachine_util:media_type_to_detail(CT),</div><div> {[{MT, accept_content}], ReqData, Context}.</div><div><br></div><div>accept_content(ReqData, Context) -></div>
<div> Path = "/some/path/",</div><div> stream_data_to_disk(wrq:stream_req_body(ReqData, ?CHUNK), Path, Start);</div><div><br></div><div>stream_data_to_disk(Data, File, Position) -></div><div> {ok, FH} = file:open(File, [binary, raw, write, read]),</div>
<div> stream_data_to_disk(Data, FH).</div><div><br></div><div>stream_data_to_disk({Hunk, done}, File) -></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>file:write(File, Hunk),</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>file:close(File);</div>
<div><br></div><div>stream_data_to_disk({Hunk, Next}, File) -></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>file:write(File, Hunk),</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>stream_data_to_disk(Next(), File).</div>
</div><div><br></div><div>Client:</div><div><br></div><div><div>read_file({file, Name, Pid, RemotePos}) -></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>case file:open(Name, [read, raw, binary]) of</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>{ok, FD} -></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>Context = utils:get_md5_context(FD, RemotePos),</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>read_file({fd, FD, Pid, Context});</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>{error, _} -></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>eof</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>end;</div>
<div>read_file({fd, FD, Pid, Cntx}) -></div><div><span class="Apple-tab-span" style="white-space: pre; "> </span>case file:read(FD, ?CHUNK) of</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>{ok, Data} -></div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>NewContext = crypto:md5_update(Cntx, Data),</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>{ok, Data, {fd, FD, Pid, NewContext}};</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>eof -></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>file:close(FD),</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>Pid ! {hash, crypto:md5_final(Cntx)},</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>eof;</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>_ -></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>eof</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>end.</div></div><div><br></div><div>transmit(FileName, Url, _, Size) -></div><div>Pid = self(),</div><div>RemotePos = get_size(filename:basename(FileName), Url),</div>
<div><div>httpc:request(put,</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>{url(Url, filename:basename(FileName)), </div><div><span class="Apple-tab-span" style="white-space:pre"> </span> [range_header(RemotePos, Size - RemotePos, Size)], </div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span> "application/octet-stream", </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>{chunkify, fun read_file/1, {file, FileName, Pid, RemotePos}}},[{ssl, [{reuse_sessions, true}]}], [{body_format, binary}] )</div>
</div>