[erlang-questions] What's the proper scenario for ecio?

[songlu cai]

Hi,



I really appreciate the the ecio feature on OTP 17.4(for lower latency and
smoother management

of IO), I run the test for two scenes, one for short tcp connection, one
for keepalive tcp connection.

In order to measure effects of ecio, I make a wrapper for
prepare_for_sys_schedule:



static ERTS_INLINE *int*

prepare_for_sys_schedule_wrapper(ErtsSchedulerData *esdp, *int*
non_blocking)

{

        *int* r = prepare_for_sys_schedule(esdp, non_blocking);

        *int* *index* = non_blocking + erts_eager_check_io*2;

        *if* (r) {

                erts_no_atomic32_inc(&(schedule_statistics[*index*]));

}

        erts_no_atomic32_inc(&(schedule_statistics[4]));



        *if* (schedule_statistics[4] % 10000 == 0) {

                erts_print(ERTS_PRINT_STDOUT, NULL, "counts:%d %d %d %d
%d\n", schedule_statistics[4], schedule_statistics

[0], schedule_statistics[1], schedule_statistics[2],
schedule_statistics[3]);

        }

        *return* r;

}



ERTS_CIO_EXPORT(erts_check_io)(*int* do_wait)

{

    erts_no_atomic32_inc(&(check_count[do_wait]));

    erts_no_atomic32_inc(&(check_count[3]));

    *if* (check_count[3] % 10000 == 0) {

        erts_print(ERTS_PRINT_STDOUT, NULL, "check:%d %d %d\n",
check_count[3], check_count[0], check_count[1]);

    }

}



Test A:

Test for http (short tcp connection), a http server runs on erl, a client
makes stress test with apache ab cmd: ./ab -c 60 -n 100000
http://10.10.10.10:8888/

Test A1:

./bin/erl +K true +h 99999  +P 99999 -secio true -smp enable -s ehttpd

Result:

counts:30700000 0 0 1465043 2728782

check:17860000 14405477 3454523

prepare_for_sys_schedule=1 ratio (1465043 + 2728782)/30700000 = 13.6%



Compared with

Test A2:

./bin/erl +K true +h 99999  +P 99999 -secio false -smp enable -s ehttpd

counts:36110000 1433811 919525 0 0

check:16070000 12576257 3493743

prepare_for_sys_schedule=1 ratio (1433811 + 919525)/36110000 = 6.5%



Check_io_count ratio: A1 check_io / A2 check_io = 17860000/16070000  = 1.11

Time Cost: same (diff < 0.5%)

QPS: same (diff < 0.5%)

CPU: A1 62% per CPU, A2 64% per CPU



Test B:

Test for tcp keepalive connection, a proxy server runs on erl, a client
make stress test with mysql request.

Test B1:

beam.smp -secio true -sbt db -sbwt very_short -swt low -sfwi 500 -MBmmsbc
100 -MHmmsbc 100 -MBmmmbc 100 -MHmmmbc 100 -MMscs 1024 -MBsmbcs 10240
-MHsbct 2048 -hmbs 46422 -hms 2586 -zdbbl 8192 -W w -P 1000000 -A 16 -K
true -d -e 50000 -Q 1000000 -sct
L0T0C0P0N0:L1T1C1P0N0:L2T2C2P0N0:L3T3C3P0N0:L4T4C4P0N0:L5T5C5P0N0:L6T6C0P1N1:L7T7C1P1N1:L8T8C2P1N1:L9T9C3P1N1:L10T10C4P1N1:L11T11C5P1N1:L12T12C0P0N0:L13T13C1P0N0:L14T14C2P0N0:L15T15C3P0N0:L16T16C4P0N0:L17T17C5P0N0:L18T18C0P1N1:L19T19C1P1N1:L20T20C2P1N1:L21T21C3P1N1:L22T22C4P1N1:L23T23C5P1N1
-swct medium -- -root /path/ -progname ** -- -home /path -- -boot /path
-mode interactive -config /path -shutdown_time 30000 -setcookie *** -name
name@REDACTED – console

Result:

*counts*:72090000 0 0 13711145 1499024

*check*:16230000 15777657 452343

prepare_for_sys_schedule=1 ratio = 21



Test B2:

beam.smp -secio false -sbt db -sbwt very_short -swt low -sfwi 500 -MBmmsbc
100 -MHmmsbc 100 -MBmmmbc 100 -MHmmmbc 100 -MMscs 1024 -MBsmbcs 10240
-MHsbct 2048 -hmbs 46422 -hms 2586 -zdbbl 8192 -W w -P 1000000 -A 16 -K
true -d -e 50000 -Q 1000000 -sct
L0T0C0P0N0:L1T1C1P0N0:L2T2C2P0N0:L3T3C3P0N0:L4T4C4P0N0:L5T5C5P0N0:L6T6C0P1N1:L7T7C1P1N1:L8T8C2P1N1:L9T9C3P1N1:L10T10C4P1N1:L11T11C5P1N1:L12T12C0P0N0:L13T13C1P0N0:L14T14C2P0N0:L15T15C3P0N0:L16T16C4P0N0:L17T17C5P0N0:L18T18C0P1N1:L19T19C1P1N1:L20T20C2P1N1:L21T21C3P1N1:L22T22C4P1N1:L23T23C5P1N1
-swct medium -- -root /path/ -progname ** -- -home /path -- -boot /path
-mode interactive -config /path -shutdown_time 30000 -setcookie *** -name
name@REDACTED – console

Result:

*counts*:72170000 13840081 1023183 0 0

*check*:15630000 15303454 326546

prepare_for_sys_schedule=1 ratio = 20.5%

Time Cost: B1 with 18 min 17 seconds, B2 with 18 min 12 seconds

CPU: B1 with around 1000%, B2 with around 1000%



I test scene B for several times, the result is nearly the same, then I
review the code and debug the key path for ecio:



*else* *if* (!ERTS_SCHEDULER_IS_DIRTY(esdp) &&

                 (fcalls > input_reductions &&

                  prepare_*for*_sys_schedule(esdp, !0))) {



I find when io task is heavy(in Test B), we barely get here, so there is
nearly no effect with +secio true.  In test A, +secio has a little effect.

I presume that +scio true only works with High CPU & Low IO scene, so we
can prevent the port task from IO starvation. Is that true? And how do you
test ecio?



Best Regards,

Songlu Cai
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150112/88e80b6c/attachment.htm>