It behaves much worse on my laptop. In smp it performs 4kB in about 5 sec. It is not issue of R13B because I'm using version 5.6.5. I have suspected issue in message queue in file:write then changed listener to<br><br>
listener(Fd,N) -><br> receive<br> {data,Data} -> Fd ! {io_request, self(), Fd, {put_chars, Data}}, listener(Fd,N+1);<br> {count,From} -> From ! {count,N}, listener(Fd,N);<br> {stop,From} -> From ! {stop,success};<br>
{io_reply, Fd, ok} -> listener(Fd, N);<br> Msg -> io:format("Unexpected message: ~p~n", [Msg]), listener(Fd, N)<br> end.<br><br>but without any effect. Than I have tried ad kernel pool and asynchronous threads (erl +K true +A 2) without any observable effect. I have also tried file:open(Filename,[append, delayed_write]). No effect.<br>
It seems that when I solve issue with message queue in testLogger:listener it happen again in file_io_server. I have end up with reasonable fast but still not good enough (it is able to fall to swapping)<br><br>-define(BS, 65536).<br>
<br>listener(Filename) -><br> case file:open(Filename,[append, raw]) of<br> {ok,Fd} -> listener(Fd,0, <<>>);<br> {error,Reason} -> error_logger:error_msg(Reason)<br> end.<br><br>listener(Fd,N, <<>> = SoFar) -><br>
receive<br> {data,Data} -> listener(Fd, N+1, iolist_to_binary(Data));<br> {count,From} -> From ! {count,N}, listener(Fd, N, SoFar);<br> {stop,From} -> From ! {stop,success};<br> Msg -> io:format("Unexpected message: ~p~n", [Msg]), listener(Fd, N, SoFar)<br>
end;<br>listener(Fd,N, SoFar) when size(SoFar) > ?BS -> file:write(Fd, SoFar), listener(Fd, N, <<>>);<br>listener(Fd,N, SoFar) -><br> receive<br> {data,Data} -><br> BinData = iolist_to_binary(Data),<br>
Bin = <<SoFar/binary, BinData/binary>>,<br> listener(Fd, N+1, Bin);<br> {count,From} -> From ! {count,N}, listener(Fd, N, SoFar);<br> {stop,From} -> file:write(Fd, SoFar), From ! {stop,success};<br>
Msg -> io:format("Unexpected message: ~p~n", [Msg]), listener(Fd, N, SoFar)<br> after 0 -><br> file:write(Fd, SoFar),<br> listener(Fd, N, <<>>)<br> end.<br><br>P.S.: Version with caching and asynchronnous calls to file_io_server instead prim_file (raw) consumes more memory.<br>
<br><div class="gmail_quote">On Fri, Apr 24, 2009 at 4:23 PM, chaitanya Chalasani <span dir="ltr"><<a href="mailto:chaitanya.chalasani@gmail.com">chaitanya.chalasani@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
Hi,<br>
<br>
I am facing performance issue with my file logger. The file logger is an event handler holding the file reference is its state. The performance of the module is way ahead in the smp disabled erl shell compared with default smp enabled erl shell.<br>
<br>
I tried ...<br>
case#1<br>
<i>Erlang R13B (erts-5.7.1) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false]</i><br>
<br>
<i>Eshell V5.7.1 (abort with ^G)</i><br>
<i>1> testLogger:start("test.log").</i><br>
<i>true</i><br>
<i>2> testLogger:write("Testing...",1000000).</i><br>
<i>{ok,done}</i><br>
<i>3></i><br>
<br>
and <br>
<br>
case#2<br>
<i>Erlang R13B (erts-5.7.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]</i><br>
<br>
<i>Eshell V5.7.1 (abort with ^G)</i><br>
<i>1> testLogger:start("test.log").</i><br>
<i>true</i><br>
<i>2> testLogger:write("Testing...",1000000).</i><br>
<i>{ok,done}</i><br>
<i>3></i><br>
<br>
<br>
What I observed was -<br>
Case 1: <br>
1. The testLogger:write("Testing...",1000000) did not return back the control to the shell until all the messages are written to log.<br>
2. The data was written at 400 KB/s<br>
<br>
Case 2:<br>
1. The testLogger:write("Testing...",1000000) return back the control to the shell with {ok, done} after a short while but the messages where being written to log even after that.<br>
2. The data was written at 4 KB/s<br>
3. I checked in the pman tool it shows current funtion as io:wait_io_mon_reply/2 and the messages in the message pool are about 880k.<br>
<br>
Below is the code which I tested.<br>
<br>
<i>start(Filename) -> register(testLogger,spawn(testLogger,listener,[Filename])).</i><br>
<br>
<i>listener(Filename) -></i><br>
<i> case file:open(Filename,[append]) of</i><br>
<i> {ok,Fd} -> listener(Fd,0);</i><br>
<i> {error,Reason} -> error_logger:error_msg(Reason)</i><br>
<i> end.</i><br>
<br>
<i>listener(Fd,N) -></i><br>
<i> receive</i><br>
<i> {data,Data} -> file:write(Fd,Data), listener(Fd,N+1);</i><br>
<i> {count,From} -> From ! {count,N}, listener(Fd,N);</i><br>
<i> {stop,From} -> From ! {stop,success}</i><br>
<i> end.</i><br>
<br>
<i>write(_Data,0) -> {ok,done};</i><br>
<i>write(Data,N) -> testLogger ! {data,Data}, write(Data,N-1).</i><br>
<br>
Is there a problem with my code or is it a known issue?<br>
<br>
<table width="100%" cellpadding="0" cellspacing="0">
<tbody><tr>
<td>
<b><font size="2"><font color="#808080">------</font></font></b><br><font color="#888888">
<b><font size="2"><font color="#808080">CHAITANYA CHALASANI</font></font></b>
</font></td>
</tr>
</tbody></table>
</div>
<br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://www.erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://www.erlang.org/mailman/listinfo/erlang-questions</a><br></blockquote></div><br><br clear="all"><br>-- <br>--Hynek (Pichi) Vychodil<br>
<br>Analyze your data in minutes. Share your insights instantly. Thrill your boss. Be a data hero!<br>Try Good Data now for free: <a href="http://www.gooddata.com">www.gooddata.com</a><br>