[erlang-questions] remsh failing silently

[Ben Hood]

Hi Mikael,

Many thanks for taking the time to look into this, very much appreciated.

To make it easier to see what is going on, I've pushed my Makefile here:

https://github.com/0x6e6562/otp-ppc64le/blob/master/otp/Makefile

This set of Makefiles first builds openssl and ncurses statically, and
then attempts to cross compile OTP and link those two libraries.

The cross compiler is provided by the IBM Advanced Toolchain:
https://developer.ibm.com/linuxonpower/advance-toolchain/

The glibc runtime for the target machine is provided by the IBM
Advanced Toolchain as well.

On Wed, Jan 2, 2019 at 1:07 PM Mikael Pettersson <mikpelinux@REDACTED> wrote:
> > There seems to be configure options to link openssl statically, but3. If you want ncurses and openssl statically linked I'd build them with --disable-shared in the first place, alternatively (temporarily) remove their .so files from the cross compiler's libs.

It doesn't look like the ncurses/openssl builds are producing any *.so files.

But I'll look into the disable-shared flag for good measure.

ATM the ncurses build produces what seems to be a static lib:

$ readelf -h ncurses/dist/lib/libncurses.a

File: ncurses/dist/lib/libncurses.a(version.o)
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           PowerPC64
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          752 (bytes into file)
  Flags:                             0x2, abiv2
  Size of this header:               64 (bytes)
  Size of program headers:           0 (bytes)
  Number of program headers:         0
  Size of section headers:           64 (bytes)
  Number of section headers:         13
  Section header string table index: 12

> This doesn't say how ncurses was build, but I note a few things:

Fair point. I was trying to be too minimalistic - for reference I've
pushed the Makefile(s) here: https://github.com/0x6e6562/otp-ppc64le

> 1. --enable-native-libs is meaningless with --disable-hipe

OK, that's good to know. I think this is left over from me trying to
get HiPE compilation working with the cross compiler, but I abandoned
this - walk before you can run.

> 2. All those overrides of LDFLAGS, LIBS, CFLAGS etc look unnecessary.

By trial and error, this was the only way I found to specify the IBM
Advance Toolchain version of gcc, ld, etc.

> If I was doing this, I'd build a self-contained cross-compiler with
> binutils, gcc, glibc, ncurses, and openssl in $CROSS (e.g.
> /opt/cross-ppc64-linux/), put $CROSS/bin first in PATH, then build OTP
> following the procedure in HOWTO/INSTALL-CROSS.md section "Building
> With configure/make Directly".

Ah, so you build a special directory of all of the cross compiled
dependencies, put the bin directory first on the PATH and then this
will make sure that the subsequent OTP build picks up those specific
versions of gcc etc?

> 3. You might want to build ncurses and openssl with --disable-shared
> to ensure that no dynamic linking is attempted.

OK - I'll make this explicit in the ncurses Makefile.

> 4. If you `ldd beam.smp` on the target, does it list any unexpected
> shared libraries?  And even if say libc is dynamically linked, is it
> the same version as used in the cross compilation environment?

This is the ldd output for beam.smp - not sure whether it shows
anything surprising.

linux-vdso64.so.1 =>  (0x00003fff84c30000)
libutil.so.1 => /opt/at12.0/lib64/power8/libutil.so.1 (0x00003fff84c00000)
libdl.so.2 => /opt/at12.0/lib64/power8/libdl.so.2 (0x00003fff84bd0000)
libm.so.6 => /opt/at12.0/lib64/power8/libm.so.6 (0x00003fff84a70000)
libpthread.so.0 => /opt/at12.0/lib64/power8/libpthread.so.0 (0x00003fff84a20000)
libc.so.6 => /opt/at12.0/lib64/power8/libc.so.6 (0x00003fff847d0000)
/opt/at12.0/lib64/ld64.so.2 => /lib64/ld64.so.2 (0x0000000051060000)

I wonder if there is an executable test utility in the ncurses build
to make sure the library itself is compiled properly.

Ben