[erlang-questions] Desperately looking for official Erlang tests results

[Loïc Hoguin]

On 03/02/2017 01:37 PM, Björn Gustavsson wrote:
> Did you run Erlang from the git repository/source
> tree or did you do "make release" and run it from
> there?
>
> A few tests cases will fail if you have not done
> "make release". (One example is
> xref_SUITE:md/1 in tools.)

When should we do this exactly?

It is not mentioned here: https://github.com/erlang/otp/wiki/Running-tests

Cheers,

> I think that the best way for us to help you is
> if you can share the names of the failed
> test cases in the following test suites:
>
> emulator_test
> stdlib_test
> kernel_test
>
> We should be able to tell you which ones
> that are most likely real bugs, and which
> ones that fail because of envioronmental
> issues.
>
> /Björn
>
> On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony <tony.reix@REDACTED> wrote:
>> Hi,
>>
>> In order to understand which Erlang tests and features do not work fine
>> within our port of version 19.2 on AIX, we need some "official" Erlang tests
>> results for current version (19.2.3).
>> However, searching on Erlang web-site, I was unable to find such tests
>> results.
>>
>> So, I have built and tested Erlang on 3 environments: AIX 6.1,
>> Fedora25/PPC64, and Fedora25/x86_64.
>> I hope I've correctly ran the tests, but I'm not sure.
>>
>> In short, using nearly the same procedure (use Fedora .spec file), but
>> without HiPE-SMP-Threads for AIX, I've got:
>>
>>                  OK  FAIL   SKIP   TOTAL
>> ------------------------------------------
>> AIX 6.1      10,838   204  2,991  14,033   (no HiPE-SMP-Threads !)
>> FC25/PPC64   12,376   193  3,608  16,177
>> FC25/x86_64  12,407   288  3,475  16,170
>>
>> I am really surprised by:
>>   - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all
>> tests !!
>>   - how few failures I have on AIX, comparable to the amount of failures on
>> FC25/x86_64.
>>         (However:   - when trying with SMP and/or HiPE, I had hangs, so I'm
>> first building/testing without them
>>                             - we applied several AIX-specific patches)
>>  - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
>> (However: there are some more SKIPped tests: 133)
>>
>> Since result files contains about 600,000 lines, it is very difficult for me
>> to understand where/what are the issues.
>> I think we have understood that, in many cases, some environment is
>> missing/failing and thus the test is skipped.
>>
>> I'd like to get some help about:
>>   - checking that I'm doing the Erlang configuration, build, and tests
>> correctly,
>>   - finding some "official" Erlang tests results on Fedora / x86_64
>>   - how to analyze these results and find the real issues.
>>
>> I have attached to this email the index.html file produced on
>> Fedora25/x86_64 :
>>
>> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>>
>> Thanks
>>
>> Cordialement,
>>
>> Tony Reix
>>
>> Bull - ATOS
>> IBM Coop Architect & Technical Leader
>> Office : +33 (0) 4 76 29 72 67
>> 1 rue de Provence - 38432 Échirolles - France
>> www.atos.net
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
>

-- 
Loïc Hoguin
https://ninenines.eu