[erlang-questions] Low disk logging performance in SMP

[Hynek Vychodil]

I have no problem with mine version, it does about 5MB/s for me.

On Fri, Apr 24, 2009 at 9:08 PM, Per Melin <per.melin@REDACTED> wrote:

> Per Melin:
> > Anders Nygren:
> >> Per Melin:
> >>> It is known that once you queue up a few tens of thousand messages
> >>> into a mailbox your performance drops noticeably for doing a receive.
> >>
> >> I hope You meant when doing a SELECTIVE receive, right.
> >
> > Of course. I've been away from Erlang for too long (again). I'm really
> > sorry for the misinformation.
>
> At this point I should just be quiet, but I can't help myself.
>
> The performance characteristics of this problem makes it look like
> there's indeed a selective recieve in there somewhere. When opening
> the file in raw mode I didn't expect that. But:
>
> 1) The file:write gets faster and faster as the size of the message
> queue shrinks.
>
> 2) If you change the writer so that it waits for an ack from the
> listener between each send the problem goes away.
>
> I don't have time to learn the ins and outs of the prim_file module,
> but I did a quick trace and I suspect prim_file:drv_get_response/1
> (which does a selective receive) is the culprit.
>
> Anyway, here's a fast synchronous version of the original code:
>
> start(Filename) ->
> register(testLogger,spawn(testLogger,listener,[Filename])).
>
> listener(Filename) ->
>     case file:open(Filename,[append, raw]) of
>         {ok,Fd} -> listener(Fd,0);
>        {error,Reason} -> error_logger:error_msg(Reason)
>    end.
>
> listener(Fd,N) ->
>    receive
>         {data, From, Data} ->
>            file:write(Fd,Data),
>            From ! ack,
>             listener(Fd,N+1);
>        {count,From} -> From ! {count,N}, listener(Fd,N);
>        {stop,From} -> From ! {stop,success}
>    end.
>
> write(_Data,0) -> {ok,done};
> write(Data,N) ->
>     testLogger ! {data, self(), Data},
>    receive
>        _ -> ok
>    end,
>    write(Data,N-1).
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>



-- 
--Hynek (Pichi) Vychodil

Analyze your data in minutes. Share your insights instantly. Thrill your
boss.  Be a data hero!
Try Good Data now for free: www.gooddata.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20090424/cd8dec02/attachment.htm>