Segmentation fault in erts_printf_char (R13B04)

[Kustarto Widoyo]

Hi All,

We found that our application was crashed and core dump file was 
generated, but not found any erl_crash.dump file created.

* The application uses erlang distribution protocol that more than 50 
nodes are involving in. All nodes run on Redhat 5.3.

* We're using Erlang R13B04 64 bit built from source and the following 
patches have been applied:
	otp_src_R13B04-OTP-8475.patch
	otp_src_R13B04-OTP-8612.patch
	otp_src_R13B04-OTP-8643.patch
	otp_src_R13B04-OTP-8658.patch
	otp_src_R13B04-OTP-8661.patch
	otp_src_R13B04-OTP-8662.patch
	otp_src_R13B04-beam-break.patch
	otp_src_R13B04-emacs.patch
	otp_src_R13B04-erl_poll.patch
	otp_src_R13B04-erts_de_busy_limit.patch
	otp_src_R13B04-eunit.patch
	otp_src_R13B04-httpc-memoryleak.patch
	otp_src_R13B04-patch-etop.patch
	otp_src_R13B04-patch-odbc-oracleworkaround.patch
	otp_src_R13B04-supervisor.patch

* The erl command line we use:
$ /usr/local/gemini/ert/R13B04/lib/erlang/erts-5.7.5/bin/beam.smp -A 64 
-K true -S 0 -- -root /usr/local/gemini/ert/R13B04/lib/erlang -progname 
erl -- -home /export/home/mmssys -- -smp enable -noshell -noinput 
-noshell -sname gdss1 -kernel net_ticktime 20 -boot 
/usr/local/gemini/gdss/1.0.0/lib/app/gdss_all -config 
/usr/local/gemini/gdss/1.0.0/../var/data/node1.config -pa 
/usr/local/gemini/gdss/1.0.0/lib/app-patches -pz 
/usr/local/gemini/gdss/1.0.0/lib/app -pz 
/usr/local/gemini/gdss/1.0.0/lib -central_config 
/usr/local/gemini/gdss/1.0.0/etc/central.conf -ticket_broker_config 
/usr/local/gemini/gdss/1.0.0/etc/broker.conf


* The following is gdb and backtrace output:
------------------------
[root@REDACTED data]# gdb 
/usr/local/gemini/ert/R13B04/lib/erlang/erts-5.7.5/bin/beam.smp core.14720
GNU gdb Fedora (6.8-27.el5)
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu"...
Reading symbols from /lib64/libutil.so.1...done.
Loaded symbols for /lib64/libutil.so.1
Reading symbols from /lib64/libdl.so.2...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libm.so.6...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /usr/lib64/libncurses.so.5...done.
Loaded symbols for /usr/lib64/libncurses.so.5
Reading symbols from /lib64/libpthread.so.0...done.
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/librt.so.1...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libc.so.6...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from 
/usr/local/gemini/ert/R13B04/lib/erlang/lib/crypto-1.6.4/priv/lib/crypto_drv.so...done.
Loaded symbols for 
/usr/local/gemini/ert/R13B04/lib/erlang/lib/crypto-1.6.4/priv/lib/crypto_drv.so
Reading symbols from 
/usr/local/gemini/ert/R13B04/openssl/lib/libcrypto.so.0.9.8...done.
Loaded symbols for 
/usr/local/gemini/ert/R13B04/openssl/lib/libcrypto.so.0.9.8
Core was generated by 
`/usr/local/gemini/ert/R13B04/lib/erlang/erts-5.7.5/bin/beam.smp -A 64 
-K true -'.
Program terminated with signal 11, Segmentation fault.
[New process 14798]

snip ... snip ... snip ...

#0  0x0000000000585ea8 in erts_printf_char (fn=0x5880f0 <write_ds>, 
arg=0xb750930, c=91 '[') at common/erl_printf_format.c:840
840     common/erl_printf_format.c: No such file or directory.
         in common/erl_printf_format.c
(gdb) bt
#0  0x0000000000585ea8 in erts_printf_char (fn=0x5880f0 <write_ds>, 
arg=0xb750930, c=91 '[') at common/erl_printf_format.c:840
#1  0x000000000048c6d8 in print_term (fn=0x5880f0 <write_ds>, 
arg=0xb750930, obj=5800176, dcount=0x458f2bc8)
     at beam/erl_printf_term.c:346
#2  0x000000000048c740 in print_term (fn=0x5880f0 <write_ds>, 
arg=0xb750930, obj=<value optimized out>, dcount=0x458f2bc8)
     at beam/erl_printf_term.c:349
#3  0x000000000048c740 in print_term (fn=0x5880f0 <write_ds>, 
arg=0xb750930, obj=<value optimized out>, dcount=0x458f2bc8)
     at beam/erl_printf_term.c:349

snip ... snip ... snip ...
------------------------

* And in syslog, we found:
Jan 27 23:48:21 gds001c kernel: beam.smp[14798]: segfault at 
0000000044ef3ff8 rip 0000000000585ea8 rsp 0000000044ef4000 error 6

Please let me know, if there is anything else we have to provide.

Regards,
-- 
Kustarto Widoyo
Gemini Mobile Technologies