[erlang-questions] Desperately looking for official Erlang tests results

[REIX, Tony]

Hi Björn,

I got Erlang source from Fedora:
  # yumdownloader --source erlang
I got:
  erlang-19.2.3-1.fc25.src.rpm

It contains a .spec file, otp-OTP-19.2.3.tar.gz, and all other files (patches) needed for building Erlang.

And I built Erlang by means of:
  rpmbuild -ba erlang-19.2.3-1.fc25.spec

However, this .spec file does not build and launch the Erlang tests. I had to search Erlang documentation and find some commands.
Main ones are:
  make release_tests
  make tests
  $ERL_TOP/bin/erl -pa $TESTROOT/test_server -s ts install -s ts run all_tests -s init stop
This produced the results I provided in previous email.

Cordialement,

Tony Reix

Bull - ATOS
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net

________________________________________
De : bgustavsson@REDACTED [bgustavsson@REDACTED] de la part de Björn Gustavsson [bjorn@REDACTED]
Envoyé : jeudi 2 mars 2017 13:37
À : REIX, Tony
Cc : erlang-questions@REDACTED; GIRARDET, JEAN
Objet : Re: [erlang-questions] Desperately looking for official Erlang tests results

Did you run Erlang from the git repository/source
tree or did you do "make release" and run it from
there?

A few tests cases will fail if you have not done
"make release". (One example is
xref_SUITE:md/1 in tools.)

I think that the best way for us to help you is
if you can share the names of the failed
test cases in the following test suites:

emulator_test
stdlib_test
kernel_test

We should be able to tell you which ones
that are most likely real bugs, and which
ones that fail because of envioronmental
issues.

/Björn

On Thu, Mar 2, 2017 at 12:23 PM, REIX, Tony <tony.reix@REDACTED> wrote:
> Hi,
>
> In order to understand which Erlang tests and features do not work fine
> within our port of version 19.2 on AIX, we need some "official" Erlang tests
> results for current version (19.2.3).
> However, searching on Erlang web-site, I was unable to find such tests
> results.
>
> So, I have built and tested Erlang on 3 environments: AIX 6.1,
> Fedora25/PPC64, and Fedora25/x86_64.
> I hope I've correctly ran the tests, but I'm not sure.
>
> In short, using nearly the same procedure (use Fedora .spec file), but
> without HiPE-SMP-Threads for AIX, I've got:
>
>                  OK  FAIL   SKIP   TOTAL
> ------------------------------------------
> AIX 6.1      10,838   204  2,991  14,033   (no HiPE-SMP-Threads !)
> FC25/PPC64   12,376   193  3,608  16,177
> FC25/x86_64  12,407   288  3,475  16,170
>
> I am really surprised by:
>   - the BIG amount of SKIPped tests on FedoraCore 25 on x86_64 : 23% of all
> tests !!
>   - how few failures I have on AIX, comparable to the amount of failures on
> FC25/x86_64.
>         (However:   - when trying with SMP and/or HiPE, I had hangs, so I'm
> first building/testing without them
>                             - we applied several AIX-specific patches)
>  - having LESS failures on Fedora25/PPC64 than on Fedora25/x86_64 .
> (However: there are some more SKIPped tests: 133)
>
> Since result files contains about 600,000 lines, it is very difficult for me
> to understand where/what are the issues.
> I think we have understood that, in many cases, some environment is
> missing/failing and thus the test is skipped.
>
> I'd like to get some help about:
>   - checking that I'm doing the Erlang configuration, build, and tests
> correctly,
>   - finding some "official" Erlang tests results on Fedora / x86_64
>   - how to analyze these results and find the real issues.
>
> I have attached to this email the index.html file produced on
> Fedora25/x86_64 :
>
> /root/rpmbuild/BUILD/otp-OTP-19.2.3/release/tests/test_server/index.html
>
> Thanks
>
> Cordialement,
>
> Tony Reix
>
> Bull - ATOS
> IBM Coop Architect & Technical Leader
> Office : +33 (0) 4 76 29 72 67
> 1 rue de Provence - 38432 Échirolles - France
> www.atos.net
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>



--
Björn Gustavsson, Erlang/OTP, Ericsson AB