[erlang-patches] Make Erlang compile on recent FreeBSD releases (long)

Giacomo Olgeni g.olgeni@REDACTED
Sat Nov 1 23:49:07 CET 2008


(I'm reposting as the first attempt didn't show up on the list...)

Hello,

I'm following up to a previous post (http://tinyurl.com/5dtkfb) about 
FreeBSD support.

The FreeBSD port of Erlang has been collecting patches for a while. 
Some of them are critical build fixes without which it is not possible 
to compile Erlang on a recent FreeBSD installation (I'm using 7.0 
here), so I thought it would be useful to sort out all the patches by 
running through an unpatched build of R12B-4 and checking what happens 
without them. Most of the patches were actually obtained from 
erlang-patches.

All the patches may be downloaded from here:

http://www.freebsd.org/cgi/cvsweb.cgi/ports/lang/erlang/files/

The port sets --enable-threads and --enable-dynamic-ssl-lib as 
arguments for running the configure tool.

The first thing to go wrong is the PCRE library. Apparently, Erlang 
ships with its own version of PCRE which is not compatible to the PCRE 
version from the FreeBSD ports (7.8 as of today). Thus, erl_bif_re.c 
will fail to compile:

beam/erl_bif_re.c: In function 'cleanup_restart_context':
beam/erl_bif_re.c:498: warning: implicit declaration of function 
'pcre_free_restart_data'
beam/erl_bif_re.c: In function 're_run_3':
beam/erl_bif_re.c:921: error: 'PCRE_EXTRA_LOOP_LIMIT' undeclared (first use in 
this function)
beam/erl_bif_re.c:921: error: (Each undeclared identifier is reported only once
beam/erl_bif_re.c:921: error: for each function it appears in.)
beam/erl_bif_re.c:923: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:926: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:927: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:930: error: 'pcre_extra' has no member named 'restart_data'
beam/erl_bif_re.c:931: error: 'pcre_extra' has no member named 'restart_flags'
beam/erl_bif_re.c:932: error: 'pcre_extra' has no member named 
'loop_counter_return'
beam/erl_bif_re.c:972: error: 'PCRE_ERROR_LOOP_LIMIT' undeclared (first use in 
this function)
beam/erl_bif_re.c: In function 're_exec_trap':
beam/erl_bif_re.c:1023: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:1026: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:1027: error: 'pcre_extra' has no member named 'loop_limit'
beam/erl_bif_re.c:1029: error: 'pcre_extra' has no member named 
'loop_counter_return'
beam/erl_bif_re.c:1030: error: 'pcre_extra' has no member named 'restart_data'
beam/erl_bif_re.c:1031: error: 'pcre_extra' has no member named 'restart_flags'
beam/erl_bif_re.c:1039: error: 'PCRE_ERROR_LOOP_LIMIT' undeclared (first use in 
this function)
gmake[3]: *** [obj/i386-portbld-freebsd7.0/opt/plain/erl_bif_re.o] Error 1
gmake[3]: Leaving directory `/usr/storage/erlang/work/otp_src_R12B-4/erts/emulator'

Part of patch-erts-emulator-Makefile.in deals with this by sorting the 
$(CC) command line flag so that the local $(INCLUDES) show up as the 
first option.

--- erts/emulator/Makefile.in.orig
+++ erts/emulator/Makefile.in
@@ -544,7 +546,7 @@
  endif

  $(OBJDIR)/%.o: beam/%.c
-       $(CC) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) $(INCLUDES) -c $< -o $@
+       $(CC) $(INCLUDES) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) -c $< -o $@

  else

This allows to get past the PCRE issue.

Then you will probably bump into gethostbyname_r, which is called using
incompatible parameters.

connect/ei_resolve.c: In function 'ei_gethostbyname_r':
connect/ei_resolve.c:629: warning: passing argument 5 of 'gethostbyname_r' from 
incompatible pointer type
connect/ei_resolve.c:629: error: too few arguments to function 
'gethostbyname_r'
gmake[4]: *** 
[/usr/storage/erlang/work/otp_src_R12B-4/lib/erl_interface/obj.mt/i386-portbld-freebsd7.0/ei_resolve.o] 
Error 1
gmake[4]: Leaving directory 
`/usr/storage/erlang/work/otp_src_R12B-4/lib/erl_interface/src'
gmake[3]: *** [opt] Error 2
gmake[3]: Leaving directory 
`/usr/storage/erlang/work/otp_src_R12B-4/lib/erl_interface/src'
gmake[2]: *** [opt] Error 2
gmake[2]: Leaving directory 
`/usr/storage/erlang/work/otp_src_R12B-4/lib/erl_interface'
gmake[1]: *** [opt] Error 2
gmake[1]: Leaving directory `/usr/storage/erlang/work/otp_src_R12B-4/lib'
gmake: *** [libs] Error 2
*** Error code 2

The patch in patch-lib_erl__interface_src_connect_ei__resolve.c sorts this out.

--- lib/erl_interface/src/connect/ei_resolve.c.orig
+++ lib/erl_interface/src/connect/ei_resolve.c
@@ -621,7 +621,8 @@

    return result;
  #else
-  return gethostbyname_r(name,hostp,buffer,buflen,h_errnop);
+  struct hostent *dummy;
+  return gethostbyname_r(name,hostp,buffer,buflen,&dummy,h_errnop);
  #endif
  #endif
  #endif

If you rebuild with these patches, memsup.c hits you next:

memsup.c: In function 'get_extended_mem':
memsup.c:456: error: void value not ignored as it ought to be

patch-lib_os__mon_c__src_memsup.c will fix it:

--- lib/os_mon/c_src/memsup.c.orig
+++ lib/os_mon/c_src/memsup.c
@@ -404,7 +404,7 @@
  #endif

  #if defined(BSD4_4)
-static void
+static int
  get_extended_mem_bsd4(memory_ext *me) {
      struct vmtotal vt;
      long pgsz;

The next issue is about odbcserver.c:

odbcserver.c: In function 'connect_to_erlang':
odbcserver.c:1625: error: storage size of 'sin' isn't known

patch-lib_odbc_c_src_odbcserver.c includes the missing header:

--- lib/odbc/c_src/odbcserver.c.orig    Sat Jun 25 17:34:20 2005
+++ lib/odbc/c_src/odbcserver.c Sat Jun 25 17:34:33 2005
@@ -107,6 +107,7 @@
  #include <sys/socket.h>
  #include <sys/uio.h>
  #include <netdb.h>
+#include <netinet/in.h>
  #endif

  #include "ei.h"

These patches will at least get you through the build.

Now, let's try to run "gmake install".

gmake[6]: Entering directory 
`/usr/storage/erlang/work/otp_src_R12B-4/lib/crypto/c_src'
/usr/bin/install -c -o root -g wheel -d 
/usr/local/lib/erlang/lib/crypto-1.5.2.1/priv/obj
/usr/bin/install -c -o root -g wheel -d 
/usr/local/lib/erlang/lib/crypto-1.5.2.1/priv/lib
install  -s -o root -g wheel -m 555 ../priv/Makefile 
/usr/local/lib/erlang/lib/crypto-1.5.2.1/priv/obj
strip: /usr/local/lib/erlang/lib/crypto-1.5.2.1/priv/obj/Makefile: File format 
not recognized
install: wait: No such file or directory
gmake[6]: *** [release_spec] Error 70

The problem here is that Makefile is treated like an executable, so
INSTALL_PROGRAM tries to run the strip command on it, which fails because
there's no executable to strip.

patch-lib_crypto_c__src_Makefile.in switches INSTALL_PROGRAM to INSTALL_DATA:

--- lib/crypto/c_src/Makefile.in.orig   Thu Oct  7 12:11:06 2004
+++ lib/crypto/c_src/Makefile.in        Thu Oct  7 12:11:07 2004
@@ -121,7 +121,7 @@
  release_spec: opt
         $(INSTALL_DIR) $(RELSYSDIR)/priv/obj
         $(INSTALL_DIR) $(RELSYSDIR)/priv/lib
-       $(INSTALL_PROGRAM) $(DRV_MAKEFILE) $(RELSYSDIR)/priv/obj
+       $(INSTALL_DATA) $(DRV_MAKEFILE) $(RELSYSDIR)/priv/obj
         $(INSTALL_PROGRAM) $(OBJS) $(RELSYSDIR)/priv/obj
         $(INSTALL_PROGRAM) $(DYN_DRIVER) $(RELSYSDIR)/priv/lib

With this last patch you will be able to install a working Erlang.

However, there are still some issues with SCTP support.

I added "--enable-sctp" to the configure parameters, but then...

configure: configuring in erts
configure: running /bin/sh 
'/usr/storage/erlang/work/otp_src_R12B-4/erts/configure' --prefix=/usr/local 
'--enable-threads' '--enable-dynamic-ssl-lib' '--enable-sctp' 
'--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/info/' 
'--build=i386-portbld-freebsd7.0' 'CC=cc' 'CFLAGS=-O2 -fno-strict-aliasing 
-pipe -I/usr/local/include' 'LDFLAGS= -L/usr/local/lib' 
'build_alias=i386-portbld-freebsd7.0' --cache-file=/dev/null 
--srcdir=/usr/storage/erlang/work/otp_src_R12B-4/erts
[...]
checking for sys/devpoll.h... no
test: xyes: unexpected operator
checking valgrind/valgrind.h usability... no
[...]

If you look into erts/configure, you'll see that the test for enable_sctp
lies exactly between the devpoll and valgrind checks:

if test "x$enable_sctp" == "xyes" ; then
     echo "$as_me:$LINENO: checking for netinet/sctp.h" >&5
echo $ECHO_N "checking for netinet/sctp.h... $ECHO_C" >&6
if test "${ac_cv_header_netinet_sctp_h+set}" = set; then
   echo $ECHO_N "(cached) $ECHO_C" >&6
else
   cat >conftest.$ac_ext <<_ACEOF

Notice the "==" operator, which should actually be "=".

Let's patch it in patch-erts_configure:

--- erts/configure.orig
+++ erts/configure
@@ -10620,7 +10620,7 @@



-if test "x$enable_sctp" == "xyes" ; then
+if test "x$enable_sctp" = "xyes" ; then
      echo "$as_me:$LINENO: checking for netinet/sctp.h" >&5
  echo $ECHO_N "checking for netinet/sctp.h... $ECHO_C" >&6
  if test "${ac_cv_header_netinet_sctp_h+set}" = set; then

...and retry:

checking for sys/devpoll.h... no
checking for netinet/sctp.h... yes
checking for sctp_bindx in -lsctp... no
checking whether SCTP_UNORDERED is declared... yes
checking whether SCTP_ADDR_OVER is declared... yes
checking whether SCTP_ABORT is declared... yes
checking whether SCTP_EOF is declared... yes
checking whether SCTP_SENDALL is declared... yes
checking whether SCTP_ADDR_CONFIRMED is declared... yes
checking for struct sctp_paddrparams.spp_pathmtu... yes
checking for struct sctp_paddrparams.spp_sackdelay... no
checking for struct sctp_paddrparams.spp_flags... yes
checking for struct sctp_remote_error.sre_data... yes
checking for struct sctp_send_failed.ssh_data... no
checking valgrind/valgrind.h usability... no
checking valgrind/valgrind.h presence... no

Here we made some progress, however the check for sctp_bindx in -lsctp
causes "WITH_SCTP=" to be written in 
erts/emulator/i386-portbld-freebsd7.0/Makefile,
thus disabling SCTP entirely.

According to SCTP_BINDX(3), sctp_bindx is found in -lc rather than -lsctp
on FreeBSD.

So, the following patch is added to patch-erts_configure:

--- erts/configure.orig
+++ erts/configure
@@ -10679,7 +10679,6 @@
    echo $ECHO_N "(cached) $ECHO_C" >&6
  else
    ac_check_lib_save_LIBS=$LIBS
-LIBS="-lsctp  $LIBS"
  cat >conftest.$ac_ext <<_ACEOF
  /* confdefs.h.  */
  _ACEOF

However this should probably be fixed in configure.in, together with the
(now incorrect) log message.

This time we get the correct result:

checking for netinet/sctp.h... yes
>>> checking for sctp_bindx in -lsctp... yes <<<
checking whether SCTP_UNORDERED is declared... yes

If we rebuild, we are stopped again here:

[...] obj/i386-portbld-freebsd7.0/opt/plain/efile_drv.o 
obj/i386-portbld-freebsd7.0/opt/plain/inet_drv.o 
obj/i386-portbld-freebsd7.0/opt/plain/zlib_drv.o 
obj/i386-portbld-freebsd7.0/opt/plain/ram_file_drv.o 
obj/i386-portbld-freebsd7.0/opt/plain/ttsl_drv.o -lutil -lm   -lncurses 
-L../lib/internal/i386-portbld-freebsd7.0 
/usr/storage/erlang/work/otp_src_R12B-4/erts/emulator/zlib/obj/i386-portbld-freebsd7.0/opt/libz.a 
/usr/storage/erlang/work/otp_src_R12B-4/erts/emulator/pcre/obj/i386-portbld-freebsd7.0/opt/libepcre.a 
-lsctp -lethread -lpthread -lerts_internal_r
/usr/bin/ld: cannot find -lsctp

patch-erts-emulator-Makefile.in is then modified to avoid looking for
libsctp on FreeBSD.

--- erts/emulator/Makefile.in.orig
+++ erts/emulator/Makefile.in
@@ -265,8 +265,10 @@
  endif
  WITH_SCTP=@WITH_SCTP@
  ifdef WITH_SCTP
+ifneq ($(findstring freebsd,$(TARGET)),freebsd)
  LIBS += -lsctp
  endif
+endif

  ORG_THR_LIBS=@EMU_THR_LIBS@
  THR_LIB_NAME=@EMU_THR_LIB_NAME@

After the last patch, it appears that SCTP is working.

Erlang (BEAM) emulator version 5.6.4 [source] [async-threads:0] [hipe] 
[kernel-poll:false]

Eshell V5.6.4  (abort with ^G)
1> gen_sctp:open ().
{ok,#Port<0.96>}
2>

On the website above you will find more patches which are not critical to
the build. Specifically, there's a patch named for erl_process_dump.c
(which was obtained from this mailing list - credits due to whoever posted
it!) that fixes a potential crash, so I was wondering if it would be
suitable for inclusion in the next release.

--- erts/emulator/beam/erl_process_dump.c.orig
+++ erts/emulator/beam/erl_process_dump.c
@@ -399,9 +399,11 @@
  static void
  dump_externally(int to, void *to_arg, Eterm term)
  {
-    byte sbuf[1024]; /* encode and hope for the best ... */
-    byte* s;
-    byte* p;
+    byte *sbuf, *s, *p;
+    unsigned size;
+
+    size = encode_size_struct(term, TERM_TO_BINARY_DFLAGS);
+    sbuf = (byte *) erts_alloc(ERTS_ALC_T_TMP, size);

      s = p = sbuf;
      erts_to_external_format(NULL, term, &p, NULL, NULL);
@@ -409,4 +411,6 @@
      while (s < p) {
  	erts_print(to, to_arg, "%02X", *s++);
      }
+
+    erts_free(ERTS_ALC_T_TMP, (void *) sbuf);
  }

I would also suggest that the next Erlang releases are tested on a supported 
FreeBSD version (today 6.x should be fine) so as to make life easier for users 
who are following a customized release process and do not rely on the ports 
system to install Erlang.

Kind regards,

   Jimmy



More information about the erlang-patches mailing list