[erlang-questions] Port of Erlang (19.2) on AIX

[REIX, Tony]

Hi Andras,

Thanks for your information ! :)

Thanks for the list of minimal features (SMP, 64bit, SSL). Once I'm done with running correctly the tests with the minimal case I have now, I'll move forward and I will enable them. For now, I'm using : --disable-hipe --disable-threads --disable-smp-support --without-javac . And that already does not work since tests fail at beginning.

About the assembler issue with 1: labels, my colleague already provided me with a solution, but more complex than yours. I've asked him to explain me what the impact is to use his solution or yours.

HIPE: OK. I'll go back to it later.

SMP: As I said, I have changed some parts of the assembler, including the erts/include/internal/ppc32/atomic.h file. However, I have no idea if my changes are correct... since I'm now not using SMP.  I'll see at the 2 pages you have provided, [1] & [2] .

About "scheduler binding", I'll have to check that the way I build Erlang (by means of a FedoraCore25 modified .spec file) does generate all the appropriate flags. For now, I am in 32bit. For now, I see that some configure and Makefile (erts/emulator/powerpc-ibm-aix6.1/Makefile) files contain : DEXPORT = -Wl,-bexpall,-brtl . However, I do not see them in the traces of the build. However, "make" command hides the details. -b64 and -maix64 should come when I am in 64bit. I'll check.

Though I've been able to build Erlang till the end, the traces do not show any "./otp_build configure". However, as I said, "make" only shows a summary of what it does. And the --trace option is much too verbose.

Applications: it looks that 38 applications where built: asn1 compiler cosEvent cosEventDomain cosFileTransfer cosNotification cosProperty cosTime cosTransactions crypto diameter edoc eldap erl_docgen erl_interface eunit gs hipe ic inets kernel mnesia odbc orber os_mon otp_mibs parsetools percept public_key runtime_tools sasl snmp ssh ssl stdlib syntax_tools tools xmerl
Any important application missing ?

OTHER:

-D_LINUX_SOURCE_COMPAT : YEs !! For sure. We use it sometimes. And I forgot to try with it. I'm rebuilding now with it. Bad news: I see no improvement for my test crash.

- clock/time handling issues ? OK. I'll think of this when looking at issues.

- vsyslog : I've temporaly worked-around this. I'll go back to it later.

- OpenSSL & src.rpm : Yes. We deliver always (or nearly always) the <package>.src.rpm file on BullFreeware.

BASIC TESTING:

# erl
Erlang/OTP 19 [erts-8.2] [source]
Eshell V8.2  (abort with ^G)
1>                                                                     :) That begins to work ! :)

- SMP & core/thread number: since, for now, I'm building without SMP, I'll check this later.

- Binding schedulers: I'll go back to this later.

- erl > crypto:start() : OK : will check. Done:
    2> crypto:start().
   ok

- 3> erlang:system_info(cpu_topology).
undefined
(No SMP yet)

- test suites for the emulator. Hummm I've been able to find some instructions for running tests. No idea for now if these tests include the emulator. I'll look at this once by re-building of Erlang is done.*

-  # ./otp_build tests  : Looks like there is a mismatch in the naming:
Makefile:248: /opt/freeware/src/packages/BUILD/otp_src_19.2/make/powerpc-ibm-aix6.1.9.0/otp_ded.mk: No such file or directory

# find . -name otp_ded.mk
./make/powerpc-ibm-aix6.1/otp_ded.mk
# ln -s /opt/freeware/src/packages/BUILD/otp_src_19.2/make/powerpc-ibm-aix6.1 /opt/freeware/src/packages/BUILD/otp_src_19.2/make/powerpc-ibm-aix6.1.9.0

 With this symlink, the ./otp_build tests  command works fine, doing a lot of things, and ending without error message. However, then:
 # cd release/tests/test_server
 # erl
 ...
 1> ts:install().
 Running /opt/freeware/src/packages/BUILD/otp_src_19.2/release/tests/test_server/configure
 Env: []
 configure: error: cannot find sources (conf_vars.in) in /opt/freeware/src/packages/BUILD/otp_src_19.2/release/tests/test_server or ..
                                                                                                                                     {error,enoent}
 It seems that a file is missing.
 With : cp ./lib/common_test/test_server/conf_vars.in /opt/freeware/src/packages/BUILD/otp_src_19.2/release/tests/test_server
 that works better.
 However, the ERL ts:install(). command fails with:
    Running /opt/freeware/src/packages/BUILD/otp_src_19.2/release/tests/test_server/configure
    Env: []
    ...
    configure: error: don't know how to compile and link dynamic drivers


Salt ? We have very good salt in France ! I'll use !


Thanks a lot for your help !!!
And expect that it will take some time before I manage all the cases you have detailed for me. First, I need to have Erlang to work with the minimum, with good tests results. Then, I'll add the removed features, one by one.

Regards,

Tony

Le 15/01/2017 à 19:43, Boroska András a écrit :
Hi Tony,

On Friday I was too quick to reply and among others I missed the part that you are from Bull. :) I used your R1502 version as the starting point. Still, let me add a few more hints. Mind you my memory can fail (I was working with AIX ~ 2 years ago). Hopefully the experts will correct me if I got something wrong. View it as my 2 cents. It is not comprehensive as you will realise it soon.

> “what are the main features that must be made available ?”

AIX users tend to be large banks and similar organisations, and I imagine many of them interested in running RabbitMQ. So, based on my limited knowledge on the matter if you can start RabbitMQ that might be good enough for many. To list the minimal features I would say that SMP, 64bit, SSL are very likely. Others readers might add theirs too.

HIPE:

Hipe is mainly a performance boost for some applications, but many Erlang use cases do not require it and as Mikael said you can ignore it for now. Linux is always there for people preferring performance over AIX features. :)

SMP:

On the other hand I would argue that enabling smp support is probably a must for most use cases and it is not too difficult to achieve. One of the issues is that the AIX provided Assembler does not support the local labels syntax of the GNU Assembler. So, one either experiments with the GNU Assembler or change the labels and jumps in erts/include/internal/ppc32/atomic.h. See [1] and [2].

One can replace the label “1:\t” with “L1%=:\t” and replace the corresponding jump “bne- 1b\n\t” with “bne- L1%=\n\t”. Of course use a different label each time (E.g. L1, L2, L3, ...) This might or might not have performance implications, but definitely better than not having SMP support.

Since Power8 offers up to 96 simultaneous threads, scheduler binding might be of interest of some potential users. To achieve that replace [3] and [4] with:

#include “sys/thread.h”
bindprocessor(BINDTHREAD, thread_self(), (cpu_t) cpu)
or something similar.

64 bit and compiler/linker flags:

OBJECT_MODE=64 env variable needs to be exported obviously.

GCC needs the following flags to link the executable: -Wl,-bexpall,-brtl,-b64 -maix64

For example you might have success bootstrapping by:

CFLAGS=“-maix64 -g -O2 -D_LINUX_SOURCE_COMPAT -L/opt/freeware/lib64 -L/opt/freeware/lib” \
LDFLAGS=“-Wl,-bexpall,-brtl,-b64,-blibpath:/opt/freeware/lib64:opt/freeware/lib -maix64” \
AR=“ar -X64” \
./otp_build autoconf --with-ssl=/opt/freeware --build=<name of your build> --disable-hipe

and then the same flags for running ./otp_build configure

Optionally you can also put the flags into the respective conf files. :)

Maybe some apps will not compile, like os_mon or odbc. Tip to proceed quickly: you can skip any app by placing an empty file called SKIP in their directory. (lib/*/SKIP)

OTHER:

Note that the C standard allows the malloc(0) system call to return either zero or a pointer that cannot be dereferenced. Unixes typically return a non-zero pointer, but AIX returns 0 by default. The C interface of Erlang (erl_interface) calls malloc(0) for empty strings and checks the return value against 0 to catch allocation errors. To prevent core dumps in such cases it helps to define -D_LINUX_SOURCE_COMPAT which enforces the non-zero pointer returns. Defining it will probably not hurt and  users of C drivers will appreciate.

As I mentioned R18 reworked clock/time handling. Expect some incompatibilities in that area.

The missing vsyslog on AIX can potentially be worked around by using printf. Or perhaps you know a better solution.

As openssl versions seems to go obsolete rather quickly, expect that many users will be interested not only in the binary but also in your source rpm package when you upload to BullFreeware, so they can potentially compile and link with a different openssl version.

BASIC TESTING:

To check if core/thread number detection works start the Erlang shell and note the smp information in the first line. It should match the number of CPU threads you have. For example [smp:8:8]. See [5]

Also note the presence of [64-bit] on the same line.

Bonus (may not be critical for many) if you can bind schedulers, for example: erl +sct L0-7c0-7 +stbt db +S8:8
then erlang:system_info(scheduler_bindings). returns {0,1,2,....}

and erlang:system_info(cpu_topology). returns
[{processor,[{core,{logical,0}},{core,{logical,1}},{core,{logical,2}}, ... }]}]

While in the shell issue crypto:start(). that should return ok if linking with ssl worked.

There are also test suites for the emulator. You can run ./otp_build setup with the above CFLAGS, AR, LDFLAGS, add your bin dir to the PATH and run ./otp_build tests. Change dir to release/tests/test_server, run erl, then ts:install(), ts:run(emulator, [batch]).

Take the above information with a grain of salt as I have no access to AIX and not an expert on building Erlang. The OTP team or others might correct my mistakes.

Thank you for your work!

Best regards,

Andras

[1] https://gcc.gnu.org/ml/gcc/2005-12/msg00671.html
[2] http://stackoverflow.com/questions/3898435/labels-in-gcc-inline-assembly
[3] https://github.com/erlang/otp/blob/maint-19/erts/lib_src/common/erl_misc_utils.c#L591
[4] https://github.com/erlang/otp/blob/maint-19/erts/configure.in#L2294
[5] http://erlang.org/doc/man/erl.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20170117/8f22146d/attachment.htm>