[erlang-questions] performance issue with cowboy

[Shayan Pooya]

Almost all of the file operations in Disco use raw mode.
For the functions that do not have raw mode, we use the prim_file module
directly.

Regards.

On Sun, Nov 9, 2014 at 9:45 AM, Loïc Hoguin <essen@REDACTED> wrote:

> Sure but everything you do through file:* goes through a single Erlang
> process except if you use the 'raw' option.
>
> On 11/09/2014 04:37 PM, Shayan Pooya wrote:
>
>> Hey Danil,
>>
>> Thanks for the response. Everything is buffered in the kernel-space
>> anyway and there is no use for a user-space buffer in this case.
>>
>> On Sat, Nov 8, 2014 at 5:44 PM, Danil Zagoskin <z@REDACTED
>> <mailto:z@REDACTED>> wrote:
>>
>>     Hi!
>>
>>     Did you do any profiling?
>>
>>     Reading the patch I noticed that you have replaced single
>>     [Mochiweb's Req:stream_body + file:write] with
>>     iterative [cowboy_req:body + file:write]. So your process now writes
>>     to file (possibly blocking on it) every MTU (typically 1.5K) bytes
>>     instead of single write — about 70K syscalls per second on gigabit
>>     network.
>>     I'd suggest adding a large buffer (e.g. 1 MB) to not access the file
>>     too often.
>>
>>
>>     On Sun, Nov 9, 2014 at 12:41 AM, Shayan Pooya <shayan@REDACTED
>>     <mailto:shayan@REDACTED>> wrote:
>>
>>         Hello,
>>
>>         I have ported Disco to use cowboy instead of mochiweb. Mochiweb
>>         has been quite stable and we did not have any issues with it
>>         recently. However, cowboy uses the sendfile syscall which avoids
>>         a lot of overhead when serving static files and I thought that
>>         could lower the memory footprint of Disco when serving large
>>         files and free up some cycles. Disco+cowboy is now fully
>>         functional and all of the integration tests pass (minus some
>>         minor missing features).
>>
>>         I wanted to get a sense of the performance before making the
>>         switch and used a couple of different Disco jobs (which pressure
>>         the web-servers) to do a comparison and the Disco version with
>>         cowboy consistently takes 10 to 20 percent longer to finish. I
>>         haven't done much for optimizing the usage of cowboy yet, but
>>         the micro-benchmarks that I find on the web are contrary to what
>>         I am seeing here and I wanted to know if anyone can offer some
>>         guidance for getting better results with cowboy.
>>
>>         I am using cowboy 1.0.1 and the Cowboy patch for Disco is
>>         available at https://github.com/pooya/disco/commit/5499d696438
>>         just in case.
>>
>>         Regards.
>>
>>         _______________________________________________
>>         erlang-questions mailing list
>>         erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>>         http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>>
>>
>>     --
>>     Danil Zagoskin | z@REDACTED <mailto:z@REDACTED>
>>
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
> --
> Loïc Hoguin
> http://ninenines.eu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20141109/05a809d3/attachment.htm>