[erlang-questions] A0 outperforming A>0, why?

[Richard Carlsson]

The point of async IO threads is mainly to improve latency, not throughput.
If many processes try to do a lot of I/O simultaneously, you want to make
sure that they are never forced to wait for a long time because one of them
is doing an unusually large I/O operation. However, managing the async I/O
threads adds a little bit of overhead. If you're running a program where
the only thing that matters is the total time from start to finish, on a
single machine, and you don't care about the order in which individual
processes get to do their work, then async I/O will not be particularly
useful and mainly adds overhead. For example, in typical escripts it's a
good idea to put %%! +A0 on the second or third line of the file.

(Caveat: the OTP team have said that +A0 is not a regularly tested thing
and you should probably not use it if uptime is important. For scripts,
benchmarks etc., I see no reason to avoid it as long as you know what your
tradeoffs are.)



        /Richard

2016-09-15 16:27 GMT+02:00 Vans S <vans_163@REDACTED>:

> Running the rebar3 test suite, ct, passing A0 to the VM args is
> consistently outperforming using any value of A above 0. The test suite
> does over 1m iops.
>
> This should not be the be the case, is not the whole point of async
> threads to improve io throughput?
>
>
> The test can be setup by doing:
>
> install a package called time (apt-get install time)
>
> git clone https://github.com/erlang/rebar3.git
> rebar3 install local
> export REBAR3_ERL_ARGS="+A0 +K true"
> time -v $(LOCAL_INSTALL_REBAR3_BIN) ct
> export REBAR3_ERL_ARGS="+A1 +K true"
> time -v $(LOCAL_INSTALL_REBAR3_BIN) ct
> export REBAR3_ERL_ARGS="+A100 +K true"
> time -v $(LOCAL_INSTALL_REBAR3_BIN) ct
>
> (rebar3 passes +sbtu as well)
>
> Here is an example of some results:
>
> %A0
>     Command being timed: "/home/user/.cache/rebar3/bin/rebar3 ct"
>     User time (seconds): 203.63
>     System time (seconds): 12.12
>     Percent of CPU this job got: 128%
>     Elapsed (wall clock) time (h:mm:ss or m:ss): 2:48.44
>     Average shared text size (kbytes): 0
>     Average unshared data size (kbytes): 0
>     Average stack size (kbytes): 0
>     Average total size (kbytes): 0
>     Maximum resident set size (kbytes): 369744
>     Average resident set size (kbytes): 0
>     Major (requiring I/O) page faults: 0
>     Minor (reclaiming a frame) page faults: 2710315
>     Voluntary context switches: 215661
>     Involuntary context switches: 219472
>     Swaps: 0
>     File system inputs: 40
>     File system outputs: 1078936
>     Socket messages sent: 0
>     Socket messages received: 0
>     Signals delivered: 0
>     Page size (bytes): 4096
>     Exit status: 0
>
> %A1
>     Command being timed: "/home/user/.cache/rebar3/bin/rebar3 ct"
>     User time (seconds): 221.73
>     System time (seconds): 17.73
>     Percent of CPU this job got: 134%
>     Elapsed (wall clock) time (h:mm:ss or m:ss): 2:57.55
>     Average shared text size (kbytes): 0
>     Average unshared data size (kbytes): 0
>     Average stack size (kbytes): 0
>     Average total size (kbytes): 0
>     Maximum resident set size (kbytes): 418184
>     Average resident set size (kbytes): 0
>     Major (requiring I/O) page faults: 0
>     Minor (reclaiming a frame) page faults: 2704281
>     Voluntary context switches: 2307583
>     Involuntary context switches: 306797
>     Swaps: 0
>     File system inputs: 48
>     File system outputs: 1078928
>     Socket messages sent: 0
>     Socket messages received: 0
>     Signals delivered: 0
>     Page size (bytes): 4096
>     Exit status: 0
>
>
> %A100
>     Command being timed: "/home/user/.cache/rebar3/bin/rebar3 ct"
>     User time (seconds): 223.26
>     System time (seconds): 18.20
>     Percent of CPU this job got: 133%
>     Elapsed (wall clock) time (h:mm:ss or m:ss): 3:01.20
>     Average shared text size (kbytes): 0
>     Average unshared data size (kbytes): 0
>     Average stack size (kbytes): 0
>     Average total size (kbytes): 0
>     Maximum resident set size (kbytes): 385492
>     Average resident set size (kbytes): 0
>     Major (requiring I/O) page faults: 0
>     Minor (reclaiming a frame) page faults: 2651960
>     Voluntary context switches: 2392798
>     Involuntary context switches: 404162
>     Swaps: 0
>     File system inputs: 72
>     File system outputs: 1078936
>     Socket messages sent: 0
>     Socket messages received: 0
>     Signals delivered: 0
>     Page size (bytes): 4096
>     Exit status: 0
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160920/165039ef/attachment.htm>