[erlang-bugs] Test suite status for "master"/R15A?

Scott Lystig Fritchie fritchie@REDACTED
Thu Nov 17 06:29:21 CET 2011


Björn-Egil Dahlberg <wallentin.dahlberg@REDACTED> wrote:

b-ed> The number of failures of an entire run usually varies between
b-ed> hardware, os, environment etc. But the most I have seen have been
b-ed> nowhere near 292 failures. I think the highest i've ever seen have
b-ed> been around 200 and that was with native compiled stdlib and
b-ed> kernel and a faulty openssl. I don't have the figures in front of
b-ed> me but emulator, kernel and stdlib runs more or less clean. System
b-ed> has some undefined functions and MIB compiler error (faulty
b-ed> testcase). Corba and cos* has some issues in current master if I
b-ed> remember correctly. If inets, ssl or ssh are bonkers that usually
b-ed> mean a bad environment.

Hi, thanks for the data points.  I re-ran the full battery of tests last
night using the R15A "master" branch as of 15 Nov 2011, on a CentOS 5
64-bit virtual machine.  I'm wondering if there is either:

    1. Weird interactions between the guest OS system clock & host OS
    system clock & Erlang's timekeeping,
  or
    2. Odd timing bugs in the test suite.

... because some of last night's failures aren't repeatable, if I just
run the single test (or a single suite) by itself.  I'm going to assume
#1 right now, unless you happen to know that #2 is a real possibility?

Overall, I saw 135 failures.  A not-so-wonderful text rendering of the
top-level index.html file is included below, for the curious.

-Scott

--- snip --- snip --- snip --- snip --- snip --- 

                                                                Test Results

                                                       All test runs in "test_server"

                                                                  Test Name

   Label Test Run Started _Ok_ Failed Skipped
   (User/Auto) Missing
   Suites Node CT Log Old Runs
   tests.asn1_test -
   Wed Nov 16 2011 13:16:06 349 0 2 (2/0) 0 test_server@REDACTED CT Log none
   tests.common_test_test -
   Wed Nov 16 2011 14:44:25 116 13 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.compiler_test -
   Wed Nov 16 2011 12:23:00 850 0 12 (12/0) 0 test_server@REDACTED CT Log none
   tests.cosEventDomain_test -
   Wed Nov 16 2011 14:58:21 18 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.cosEvent_test -
   Wed Nov 16 2011 14:57:47 23 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.cosFileTransfer_test -
   Wed Nov 16 2011 14:58:38 1 0 7 (7/0) 0 test_server@REDACTED CT Log none
   tests.cosNotification_test -
   Wed Nov 16 2011 14:58:43 114 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.cosProperty_test -
   Wed Nov 16 2011 15:06:44 33 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.cosTime_test -
   Wed Nov 16 2011 15:06:47 12 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.cosTransactions_test -
   Wed Nov 16 2011 15:07:33 27 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.crypto_test -
   Wed Nov 16 2011 15:08:09 53 0 1 (1/0) 0 test_server@REDACTED CT Log none
   tests.debugger_test -
   Wed Nov 16 2011 12:33:22 179 1 28 (28/0) 0 test_server@REDACTED CT Log none
   tests.dialyzer_test -
   Wed Nov 16 2011 15:08:13 247 1 1 (1/0) 0 test_server@REDACTED CT Log none
   tests.diameter_test -
   Wed Nov 16 2011 15:38:30 63 0 4 (4/0) 5 test_server@REDACTED CT Log none
   tests.edoc_test -
   Wed Nov 16 2011 15:43:44 1 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.emulator_test -
   Wed Nov 16 2011 10:58:18 928 3 10 (10/0) 0 test_server@REDACTED CT Log none
   tests.epmd_test -
   Wed Nov 16 2011 15:43:48 26 0 2 (2/0) 0 test_server@REDACTED CT Log none
   tests.erl_interface_test -
   Wed Nov 16 2011 12:27:23 98 5 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.et_test -
   Wed Nov 16 2011 15:48:09 0 0 1 (0/1) 0 test_server@REDACTED CT Log none
   tests.eunit_test -
   Wed Nov 16 2011 15:48:10 1 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.ic_test -
   Wed Nov 16 2011 12:33:52 213 11 18 (0/18) 0 test_server@REDACTED CT Log none
   tests.inets_test -
   Wed Nov 16 2011 12:55:00 167 1 698 (2/696) 0 test_server@REDACTED CT Log none
   tests.inviso_test -
   Wed Nov 16 2011 15:48:11 6 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.jinterface_test -
   Wed Nov 16 2011 12:28:29 0 42 2 (2/0) 0 test_server@REDACTED CT Log none
   tests.kernel_test -
   Wed Nov 16 2011 11:30:28 552 4 42 (15/27) 0 test_server@REDACTED CT Log none
   tests.megaco_test -
   Wed Nov 16 2011 15:48:31 709 2 278 (36/242) 0 test_server@REDACTED CT Log none
   tests.mnesia_test -
   Wed Nov 16 2011 14:14:41 431 0 147 (147/0) 0 test_server@REDACTED CT Log none
   tests.observer_test -
   Wed Nov 16 2011 16:15:01 57 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.odbc_test -
   Wed Nov 16 2011 16:17:31 0 0 101 (0/101) 0 test_server@REDACTED CT Log none
   tests.orber_test -
   Wed Nov 16 2011 12:34:34 252 13 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.os_mon_test -
   Wed Nov 16 2011 13:44:44 24 13 3 (2/1) 0 test_server@REDACTED CT Log none
   tests.parsetools_test -
   Wed Nov 16 2011 12:33:15 29 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.percept_test -
   Wed Nov 16 2011 16:17:33 10 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.public_key_test -
   Wed Nov 16 2011 16:17:36 25 0 36 (0/36) 0 test_server@REDACTED CT Log none
   tests.reltool_test -
   Wed Nov 16 2011 16:18:28 9 6 2 (1/1) 0 test_server@REDACTED CT Log none
   tests.runtime_tools_test -
   Wed Nov 16 2011 12:30:30 49 3 1 (1/0) 0 test_server@REDACTED CT Log none
   tests.sasl_test -
   Wed Nov 16 2011 12:28:35 63 6 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.snmp_test -
   Wed Nov 16 2011 13:44:58 331 1 27 (27/0) 0 test_server@REDACTED CT Log none
   tests.ssh_test -
   Wed Nov 16 2011 16:21:50 1 0 53 (0/53) 0 test_server@REDACTED CT Log none
   tests.ssl_test -
   Wed Nov 16 2011 16:21:52 202 0 4 (3/1) 0 test_server@REDACTED CT Log none
   tests.stdlib_test -
   Wed Nov 16 2011 11:56:36 955 3 8 (6/2) 0 test_server@REDACTED CT Log none
   tests.syntax_tools_test -
   Wed Nov 16 2011 16:25:09 1 0 0 (0/0) 0 test_server@REDACTED CT Log none
   tests.system_test -
   Wed Nov 16 2011 16:28:54 57 4 1 (1/0) 0 test_server@REDACTED CT Log none
   tests.test_server_test -
   Wed Nov 16 2011 11:28:45 5 1 2 (0/2) 0 test_server@REDACTED CT Log none
   tests.tools_test -
   Wed Nov 16 2011 12:29:34 73 1 13 (13/0) 0 test_server@REDACTED CT Log none
   tests.wx_test -
   Wed Nov 16 2011 16:25:16 4 1 29 (0/29) 0 test_server@REDACTED CT Log none
   tests.xmerl_test -
   Wed Nov 16 2011 16:25:41 3569 0 240 (240/0) 0 test_server@REDACTED CT Log none
   Total     10933 135 1773 (563/1210) 5
     __________________________________________________________________________________________________________________________________

                                                   Copyright © 2011 Open Telecom Platform
                                                      Updated: Wed Nov 16 2011 16:40:19



More information about the erlang-bugs mailing list