[erlang-bugs] R13B02 on 8/16 core box: all TCP communication hangs/frozen
Rickard Green
rickard.s.green@REDACTED
Wed Nov 18 20:57:03 CET 2009
Hi Scott,
I've attached a patch that fixes a race that could cause this
situation. I haven't tested it thoroughly, but since our test suites
haven't been able to trigger this situation, I thought that it would
be best if you could test it at the same time as we are testing it.
Please, let me know if it your problem is solved or not by this patch.
If it doesn't cause any unexpected problems, it will appear in the
soon to be released R13B03.
How to apply the patch:
$ ls
otp_src_R13B02-1_OTP-8297.patch otp_src_R13B02-1.tar.gz
$ gtar -zxf otp_src_R13B02-1.tar.gz
$ patch -ZNp0 < otp_src_R13B02-1_OTP-8297.patch
patching file otp_src_R13B02-1/erts/emulator/beam/erl_process.c
$ cd otp_src_R13B02-1
$ # configure, make, and make install as usual...
Regards,
Rickard Green, Erlang/OTP, Ericsson AB
2009/11/18 Scott Lystig Fritchie <fritchie@REDACTED>:
> Hi, all. I've scoured the erlang-patches, erlang-bugs, and
> erlang-questions mailing list archives for help on this, but it doesn't
> seem to have popped up yet?
>
> While running a distributed Erlang app on a box that probably wasn't
> very busy at the time, all of its peer nodes decided that the hung box
> had timed out (some whitespace and newlines flattened):
>
> ** Node foo@REDACTED not responding **** Removing (timedout) connection **
>
> After running time of about 2 days, the VM appeared to lock up
> completely. Not possible to connect a remote shell. No "strace -f -p"
> output when connecting to the Erlang distribution TCP port reported by
> "epmd -names", nor any output when connecting to any of the TCP listener
> sockets owned by our Erlang applications inside the VM (connections via
> Telnet or "nc" would open in 0-10 seconds, usually, because most app
> listener sockets use a backlog size of 4096, but no sign of system call
> activity by the VM).
>
> The box in question is a few-months-old Dell (I can look up the model
> number if it's helpful), dual CPU, quad core + hyperthreading (or
> whatever Intel calls it today) for 16 virtual cores. If I start an
> Erlang shell without args:
>
> Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:16:16] [rq:16] [async-threads:0] [hipe] [kernel-poll:false]
>
> Eshell V5.7.3 (abort with ^G)
> (slf21203@REDACTED)1>
>
> The app is run via:
>
> env ERL_MAX_ETS_TABLES=10007 \
> erl \
> +A 64 +K true \
> -noinput -noshell \
> -kernel dist_auto_connect once \
> -kernel net_ticktime 20 \
> ...
>
> I've got output from "strace", "lsof", and a little bit of GDB stack
> backtraces from each Pthread. I've got a core file of the process via
> "gcore", so I can do other GDB stuff if it'd be helpful.
>
> I will almost certainly have at least 8 metric tons of extremely unhappy
> customer asking lots of anxious questions within 12-24 hours. Unhappy
> me. If there's other info that I can provide to the OTP team or other
> debugging parties, please let me know ASAP. And many thanks in advance!
>
> -Scott
>
> Output from "strace -f":
>
> [pid 28271] futex(0x2b3519c8d404, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28270] futex(0x2b3519c8d204, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28269] futex(0x2b3519c8d004, FUTEX_WAIT_PRIVATE, 201, NULL <unfinished ...>
> [pid 28268] futex(0x2b3519c8ce04, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28267] futex(0x2b3519c8cc04, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28266] futex(0x2b3519c8ca04, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28265] futex(0x2b3519c8c804, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28264] futex(0x2b3519c8c604, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28263] futex(0x2b3519c8c404, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28262] futex(0x2b3519c8c204, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28261] futex(0x2b3519c8c004, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28260] futex(0x2b3519c8be04, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28259] futex(0x2b3519c8bc04, FUTEX_WAIT_PRIVATE, 783259, NULL <unfinished ...>
> [pid 28258] futex(0x2b3519c8ba04, FUTEX_WAIT_PRIVATE, 4212125, NULL <unfinished ...>
> [pid 28257] futex(0x2b3519c8b804, FUTEX_WAIT_PRIVATE, 11335295, NULL <unfinished ...>
> [pid 28256] futex(0x2b3519c8b604, FUTEX_WAIT_PRIVATE, 11458135, NULL <unfinished ...>
> [pid 28255] wait4(-1, <unfinished ...>
> [pid 28254] futex(0x2aaaac92e3f4, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28253] futex(0x2aaaac92e364, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28252] futex(0x2aaaac92e2d4, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28251] futex(0x2aaaac92e244, FUTEX_WAIT_PRIVATE, 845, NULL <unfinished ...>
> [pid 28250] futex(0x2aaaac92e1b4, FUTEX_WAIT_PRIVATE, 845, NULL <unfinished ...>
> [pid 28249] futex(0x2aaaac92e124, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28248] futex(0x2aaaac92e094, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28247] futex(0x2aaaac92e004, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28246] futex(0x2aaaac92df74, FUTEX_WAIT_PRIVATE, 197345, NULL <unfinished ...>
> [pid 28245] futex(0x2aaaac92dee4, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28244] futex(0x2aaaac92de54, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28243] futex(0x2aaaac92ddc4, FUTEX_WAIT_PRIVATE, 129, NULL <unfinished ...>
> [pid 28242] futex(0x2aaaac92dd34, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28241] futex(0x2aaaac92dca4, FUTEX_WAIT_PRIVATE, 315, NULL <unfinished ...>
> [pid 28240] futex(0x2aaaac92dc14, FUTEX_WAIT_PRIVATE, 129, NULL <unfinished ...>
> [pid 28239] futex(0x2aaaac92db84, FUTEX_WAIT_PRIVATE, 152779, NULL <unfinished ...>
> [pid 28238] futex(0x2aaaac92daf4, FUTEX_WAIT_PRIVATE, 121, NULL <unfinished ...>
> [pid 28237] futex(0x2aaaac92da64, FUTEX_WAIT_PRIVATE, 121, NULL <unfinished ...>
> [pid 28236] futex(0x2aaaac92d9d4, FUTEX_WAIT_PRIVATE, 121, NULL <unfinished ...>
> [pid 28235] futex(0x2aaaac92d944, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28234] futex(0x2aaaac92d8b4, FUTEX_WAIT_PRIVATE, 1685, NULL <unfinished ...>
> [pid 28233] futex(0x2aaaac92d824, FUTEX_WAIT_PRIVATE, 152827, NULL <unfinished ...>
> [pid 28232] futex(0x2aaaac92d794, FUTEX_WAIT_PRIVATE, 127, NULL <unfinished ...>
> [pid 28231] futex(0x2aaaac92d704, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28230] futex(0x2aaaac92d674, FUTEX_WAIT_PRIVATE, 119, NULL <unfinished ...>
> [pid 28229] futex(0x2aaaac92d5e4, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28228] futex(0x2aaaac92d554, FUTEX_WAIT_PRIVATE, 127, NULL <unfinished ...>
> [pid 28227] futex(0x2aaaac92d4c4, FUTEX_WAIT_PRIVATE, 783, NULL <unfinished ...>
> [pid 28226] futex(0x2aaaac92d434, FUTEX_WAIT_PRIVATE, 129, NULL <unfinished ...>
> [pid 28225] futex(0x2aaaac92d3a4, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28224] futex(0x2aaaac92d314, FUTEX_WAIT_PRIVATE, 158751, NULL <unfinished ...>
> [pid 28223] futex(0x2aaaac92d284, FUTEX_WAIT_PRIVATE, 131, NULL <unfinished ...>
> [pid 28222] futex(0x2aaaac92d1f4, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28221] futex(0x2aaaac92d164, FUTEX_WAIT_PRIVATE, 131, NULL <unfinished ...>
> [pid 28220] futex(0x2aaaac92d0d4, FUTEX_WAIT_PRIVATE, 129, NULL <unfinished ...>
> [pid 28219] futex(0x2aaaac92d044, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28218] futex(0x2aaaac92cfb4, FUTEX_WAIT_PRIVATE, 779, NULL <unfinished ...>
> [pid 28217] futex(0x2aaaac92cf24, FUTEX_WAIT_PRIVATE, 127, NULL <unfinished ...>
> [pid 28216] futex(0x2aaaac92ce94, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28215] futex(0x2aaaac92ce04, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28214] futex(0x2aaaac92cd74, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28213] futex(0x2aaaac92cce4, FUTEX_WAIT_PRIVATE, 121, NULL <unfinished ...>
> [pid 28212] futex(0x2aaaac92cc54, FUTEX_WAIT_PRIVATE, 713, NULL <unfinished ...>
> [pid 28211] futex(0x2aaaac92cbc4, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28210] futex(0x2aaaac92cb34, FUTEX_WAIT_PRIVATE, 795, NULL <unfinished ...>
> [pid 28209] futex(0x2aaaac92caa4, FUTEX_WAIT_PRIVATE, 139, NULL <unfinished ...>
> [pid 28208] futex(0x2aaaac92ca14, FUTEX_WAIT_PRIVATE, 127, NULL <unfinished ...>
> [pid 28207] futex(0x2aaaac92c984, FUTEX_WAIT_PRIVATE, 121, NULL <unfinished ...>
> [pid 28206] futex(0x2aaaac92c8f4, FUTEX_WAIT_PRIVATE, 123, NULL <unfinished ...>
> [pid 28205] futex(0x2aaaac92c864, FUTEX_WAIT_PRIVATE, 121, NULL <unfinished ...>
> [pid 28204] futex(0x2aaaac92c7d4, FUTEX_WAIT_PRIVATE, 127, NULL <unfinished ...>
> [pid 28203] futex(0x2aaaac92c744, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28202] futex(0x2aaaac92c6b4, FUTEX_WAIT_PRIVATE, 129, NULL <unfinished ...>
> [pid 28201] futex(0x2aaaac92c624, FUTEX_WAIT_PRIVATE, 127, NULL <unfinished ...>
> [pid 28200] futex(0x2aaaac92c594, FUTEX_WAIT_PRIVATE, 133, NULL <unfinished ...>
> [pid 28199] futex(0x2aaaac92c504, FUTEX_WAIT_PRIVATE, 129, NULL <unfinished ...>
> [pid 28198] futex(0x2aaaac92c474, FUTEX_WAIT_PRIVATE, 139, NULL <unfinished ...>
> [pid 28197] futex(0x2aaaac92c3e4, FUTEX_WAIT_PRIVATE, 1881, NULL <unfinished ...>
> [pid 28196] futex(0x2aaaac92c354, FUTEX_WAIT_PRIVATE, 131, NULL <unfinished ...>
> [pid 28195] futex(0x2aaaac92c2c4, FUTEX_WAIT_PRIVATE, 131, NULL <unfinished ...>
> [pid 28194] futex(0x2aaaac92c234, FUTEX_WAIT_PRIVATE, 131, NULL <unfinished ...>
> [pid 28193] futex(0x2aaaac92c1a4, FUTEX_WAIT_PRIVATE, 127, NULL <unfinished ...>
> [pid 28192] futex(0x2aaaac92c114, FUTEX_WAIT_PRIVATE, 125, NULL <unfinished ...>
> [pid 28191] futex(0x2aaaac92c084, FUTEX_WAIT_PRIVATE, 129, NULL <unfinished ...>
> [pid 28190] futex(0x810ca4, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
> [pid 28189] read(6, [SLF: wait forever....]
>
> Output from "lsof -nP"
>
> COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
> beam.smp 28182 mmssys cwd DIR 8,3 4096 10485761 /m2h_data/data
> beam.smp 28182 mmssys rtd DIR 8,1 4096 2 /
> beam.smp 28182 mmssys txt REG 8,1 10282072 90608848 /usr/local/gemini/e
> beam.smp 28182 mmssys mem REG 8,1 139416 48660505 /lib64/ld-2.5.so
> beam.smp 28182 mmssys mem REG 8,1 1713096 48660507 /lib64/libc-2.5.so
> beam.smp 28182 mmssys mem REG 8,1 23360 48660688 /lib64/libdl-2.5.so
> beam.smp 28182 mmssys mem REG 8,1 615136 48660690 /lib64/libm-2.5.so
> beam.smp 28182 mmssys mem REG 8,1 145592 48660704 /lib64/libpthread-2
> beam.smp 28182 mmssys mem REG 8,1 53448 48660706 /lib64/librt-2.5.so
> beam.smp 28182 mmssys mem REG 8,1 18152 48660804 /lib64/libutil-2.5.
> beam.smp 28182 mmssys mem REG 8,1 380464 89141483 /usr/lib64/libncurs
> beam.smp 28182 mmssys mem REG 8,1 77360 90605001 /usr/local/gemini/e
> beam.smp 28182 mmssys mem REG 8,1 1781932 90608896 /usr/local/gemini/e
> beam.smp 28182 mmssys 0r CHR 1,3 3898 /dev/null
> beam.smp 28182 mmssys 1w CHR 1,3 3898 /dev/null
> beam.smp 28182 mmssys 2w CHR 1,3 3898 /dev/null
> beam.smp 28182 mmssys 3r 0000 0,11 0 98551559 eventpoll
> beam.smp 28182 mmssys 4r FIFO 0,6 98551560 pipe
> beam.smp 28182 mmssys 5w FIFO 0,6 98551560 pipe
> beam.smp 28182 mmssys 6r FIFO 0,6 98551561 pipe
> beam.smp 28182 mmssys 7w FIFO 0,6 98551561 pipe
> beam.smp 28182 mmssys 8u IPv4 98551569 TCP *:45315 (LISTEN)
> beam.smp 28182 mmssys 9u IPv4 98551571 TCP 127.0.0.1:56642->12
> beam.smp 28182 mmssys 10r FIFO 0,6 98551574 pipe
> beam.smp 28182 mmssys 11w REG 8,6 36288720 21397512 /foo_log/log/archiv
> beam.smp 28182 mmssys 12u IPv4 98551581 TCP *:7585 (LISTEN)
> beam.smp 28182 mmssys 13w FIFO 0,6 98551575 pipe
> beam.smp 28182 mmssys 14u IPv4 98551584 TCP *:7576 (LISTEN)
> beam.smp 28182 mmssys 15u IPv4 98551587 TCP A.B.C.140:45315
> beam.smp 28182 mmssys 16u IPv4 98551605 TCP A.B.C.140:56270
> beam.smp 28182 mmssys 17u IPv4 98551606 TCP A.B.C.140:43152
> beam.smp 28182 mmssys 18u IPv4 98551608 TCP A.B.C.140:53789
> beam.smp 28182 mmssys 19u IPv4 98551607 TCP A.B.C.140:42325
> beam.smp 28182 mmssys 20u IPv4 98551609 TCP A.B.C.140:2958-
> beam.smp 28182 mmssys 21u IPv4 98551620 TCP A.B.C.140:55959
> beam.smp 28182 mmssys 22u sock 0,5 98697328 can't identify prot
> beam.smp 28182 mmssys 23u sock 0,5 98696776 can't identify prot
> beam.smp 28182 mmssys 24u sock 0,5 98698710 can't identify prot
> beam.smp 28182 mmssys 25u sock 0,5 98695296 can't identify prot
> beam.smp 28182 mmssys 26u sock 0,5 98698083 can't identify prot
> beam.smp 28182 mmssys 27u sock 0,5 98698783 can't identify prot
> beam.smp 28182 mmssys 28u sock 0,5 98698460 can't identify prot
> beam.smp 28182 mmssys 30u sock 0,5 98698635 can't identify prot
> beam.smp 28182 mmssys 31u sock 0,5 98685423 can't identify prot
> beam.smp 28182 mmssys 32u sock 0,5 98698786 can't identify prot
> beam.smp 28182 mmssys 33u sock 0,5 98698648 can't identify prot
> beam.smp 28182 mmssys 34u sock 0,5 98698717 can't identify prot
> beam.smp 28182 mmssys 36u sock 0,5 98686061 can't identify prot
> beam.smp 28182 mmssys 37u sock 0,5 98698545 can't identify prot
> beam.smp 28182 mmssys 38u sock 0,5 98686092 can't identify prot
> beam.smp 28182 mmssys 41u sock 0,5 98698407 can't identify prot
> beam.smp 28182 mmssys 42u sock 0,5 98698722 can't identify prot
> beam.smp 28182 mmssys 43u sock 0,5 98698729 can't identify prot
>
> Output from GDB:
>
> # gdb /usr/local/gemini/ert/R13B02/lib/erlang/erts-5.7.3/bin/beam.smp ./beam.smpfoo-Nov18.0357orso.core.28182
> GNU gdb Fedora (6.8-27.el5)
> Copyright (C) 2008 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu"...
> Reading symbols from /lib64/libutil.so.1...done.
> Loaded symbols for /lib64/libutil.so.1
> Reading symbols from /lib64/libdl.so.2...done.
> Loaded symbols for /lib64/libdl.so.2
> Reading symbols from /lib64/libm.so.6...done.
> Loaded symbols for /lib64/libm.so.6
> Reading symbols from /usr/lib64/libncurses.so.5...done.
> Loaded symbols for /usr/lib64/libncurses.so.5
> Reading symbols from /lib64/libpthread.so.0...done.
> Loaded symbols for /lib64/libpthread.so.0
> Reading symbols from /lib64/librt.so.1...done.
> Loaded symbols for /lib64/librt.so.1
> Reading symbols from /lib64/libc.so.6...done.
> Loaded symbols for /lib64/libc.so.6
> Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
> Loaded symbols for /lib64/ld-linux-x86-64.so.2
> Reading symbols from /usr/local/gemini/ert/R13B02/lib/erlang/lib/crypto-1.6.1/priv/lib/crypto_drv.so...done.
> Loaded symbols for /usr/local/gemini/ert/R13B02/lib/erlang/lib/crypto-1.6.1/priv/lib/crypto_drv.so
> Reading symbols from /usr/local/gemini/ert/R13B02/openssl/lib/libcrypto.so.0.9.8...done.
> Loaded symbols for /usr/local/gemini/ert/R13B02/openssl/lib/libcrypto.so.0.9.8
> Core was generated by `/usr/local/gemini/ert/R13B02/lib/erlang/erts-5.7.3/bin/beam.smp'.
> [New process 28189]
> [New process 28190]
> [New process 28191]
> [New process 28192]
> [New process 28193]
> [New process 28194]
> [New process 28195]
> [New process 28196]
> [New process 28197]
> [New process 28198]
> [New process 28199]
> [New process 28200]
> [New process 28201]
> [New process 28202]
> [New process 28203]
> [New process 28204]
> [New process 28205]
> [New process 28206]
> [New process 28207]
> [New process 28208]
> [New process 28209]
> [New process 28210]
> [New process 28211]
> [New process 28212]
> [New process 28213]
> [New process 28214]
> [New process 28215]
> [New process 28216]
> [New process 28217]
> [New process 28218]
> [New process 28219]
> [New process 28220]
> [New process 28221]
> [New process 28222]
> [New process 28223]
> [New process 28224]
> [New process 28225]
> [New process 28226]
> [New process 28227]
> [New process 28228]
> [New process 28229]
> [New process 28230]
> [New process 28231]
> [New process 28232]
> [New process 28233]
> [New process 28234]
> [New process 28235]
> [New process 28236]
> [New process 28237]
> [New process 28238]
> [New process 28239]
> [New process 28240]
> [New process 28241]
> [New process 28242]
> [New process 28243]
> [New process 28244]
> [New process 28245]
> [New process 28246]
> [New process 28247]
> [New process 28248]
> [New process 28249]
> [New process 28250]
> [New process 28251]
> [New process 28252]
> [New process 28253]
> [New process 28254]
> [New process 28255]
> [New process 28256]
> [New process 28257]
> [New process 28258]
> [New process 28259]
> [New process 28260]
> [New process 28261]
> [New process 28262]
> [New process 28263]
> [New process 28264]
> [New process 28265]
> [New process 28266]
> [New process 28267]
> [New process 28268]
> [New process 28269]
> [New process 28270]
> [New process 28271]
> [New process 28182]
> #0 0x0000003d05a0d2cb in read () from /lib64/libpthread.so.0
> (gdb) thread apply all info stack
>
> Thread 84 (process 28182):
> #0 0x0000003d04ecc5e2 in select () from /lib64/libc.so.6
> #1 0x0000000000523290 in erts_sys_main_thread () at sys/unix/sys.c:3019
> #2 0x000000000044bb7f in erl_start (argc=34, argv=<value optimized out>)
> at beam/erl_init.c:1275
> #3 0x000000000042ee09 in main (argc=0, argv=0x0) at sys/unix/erl_main.c:29
>
> Thread 83 (process 28271):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=16, rq=0x2b3519c8d3b0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> ---Type <return> to continue, or q <return> to quit---
> Thread 82 (process 28270):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=15, rq=0x2b3519c8d1b0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 81 (process 28269):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=14, rq=0x2b3519c8cfb0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> ---Type <return> to continue, or q <return> to quit---
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 80 (process 28268):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=13, rq=0x2b3519c8cdb0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 79 (process 28267):
> ---Type <return> to continue, or q <return> to quit---
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=12, rq=0x2b3519c8cbb0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 78 (process 28266):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=11, rq=0x2b3519c8c9b0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 77 (process 28265):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=10, rq=0x2b3519c8c7b0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 76 (process 28264):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> ---Type <return> to continue, or q <return> to quit---
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=9, rq=0x2b3519c8c5b0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 75 (process 28263):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=8, rq=0x2b3519c8c3b0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> ---Type <return> to continue, or q <return> to quit---
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 74 (process 28262):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=7, rq=0x2b3519c8c1b0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 73 (process 28261):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> ---Type <return> to continue, or q <return> to quit---
> #1 0x0000000000494ab2 in sched_cnd_wait (no=6, rq=0x2b3519c8bfb0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 72 (process 28260):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=5, rq=0x2b3519c8bdb0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 71 (process 28259):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=4, rq=0x2b3519c8bbb0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 70 (process 28258):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=3, rq=0x2b3519c8b9b0)
> ---Type <return> to continue, or q <return> to quit---
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 69 (process 28257):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=2, rq=0x2b3519c8b7b0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> ---Type <return> to continue, or q <return> to quit---
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 68 (process 28256):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x0000000000494ab2 in sched_cnd_wait (no=1, rq=0x2b3519c8b5b0)
> at beam/erl_threads.h:634
> #2 0x000000000049f4c3 in schedule (p=<value optimized out>,
> calls=<value optimized out>) at beam/erl_process.c:5992
> #3 0x0000000000507bcd in process_main () at beam/beam_emu.c:1126
> #4 0x000000000049fd06 in sched_thread_func (vesdp=<value optimized out>)
> at beam/erl_process.c:3045
> #5 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #6 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #7 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 67 (process 28255):
> #0 0x0000003d05a0e03f in waitpid () from /lib64/libpthread.so.0
> #1 0x000000000052448e in child_waiter (unused=<value optimized out>)
> at sys/unix/sys.c:2828
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 66 (process 28254):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 65 (process 28253):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
> ---Type <return> to continue, or q <return> to quit---
>
> Thread 64 (process 28252):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 63 (process 28251):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 62 (process 28250):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> ---Type <return> to continue, or q <return> to quit---
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 61 (process 28249):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 60 (process 28248):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> ---Type <return> to continue, or q <return> to quit---
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 59 (process 28247):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 58 (process 28246):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> ---Type <return> to continue, or q <return> to quit---
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 57 (process 28245):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 56 (process 28244):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 55 (process 28243):
> ---Type <return> to continue, or q <return> to quit---
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 54 (process 28242):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 53 (process 28241):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 52 (process 28240):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 51 (process 28239):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> ---Type <return> to continue, or q <return> to quit---
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 50 (process 28238):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 49 (process 28237):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> ---Type <return> to continue, or q <return> to quit---
> Thread 48 (process 28236):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 47 (process 28235):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 46 (process 28234):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> ---Type <return> to continue, or q <return> to quit---
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 45 (process 28233):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 44 (process 28232):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 43 (process 28231):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 42 (process 28230):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
> ---Type <return> to continue, or q <return> to quit---
>
> Thread 41 (process 28229):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 40 (process 28228):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 39 (process 28227):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> ---Type <return> to continue, or q <return> to quit---
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 38 (process 28226):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 37 (process 28225):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> ---Type <return> to continue, or q <return> to quit---
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 36 (process 28224):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 35 (process 28223):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> ---Type <return> to continue, or q <return> to quit---
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 34 (process 28222):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 33 (process 28221):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 32 (process 28220):
> ---Type <return> to continue, or q <return> to quit---
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 31 (process 28219):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 30 (process 28218):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 29 (process 28217):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 28 (process 28216):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> ---Type <return> to continue, or q <return> to quit---
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 27 (process 28215):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 26 (process 28214):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> ---Type <return> to continue, or q <return> to quit---
> Thread 25 (process 28213):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 24 (process 28212):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 23 (process 28211):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> ---Type <return> to continue, or q <return> to quit---
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 22 (process 28210):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 21 (process 28209):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 20 (process 28208):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 19 (process 28207):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
> ---Type <return> to continue, or q <return> to quit---
>
> Thread 18 (process 28206):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 17 (process 28205):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 16 (process 28204):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> ---Type <return> to continue, or q <return> to quit---
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 15 (process 28203):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 14 (process 28202):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> ---Type <return> to continue, or q <return> to quit---
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 13 (process 28201):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 12 (process 28200):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> ---Type <return> to continue, or q <return> to quit---
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 11 (process 28199):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 10 (process 28198):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 9 (process 28197):
> ---Type <return> to continue, or q <return> to quit---
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 8 (process 28196):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 7 (process 28195):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 6 (process 28194):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 5 (process 28193):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> ---Type <return> to continue, or q <return> to quit---
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 4 (process 28192):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 3 (process 28191):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x00000000004e583f in async_main (arg=<value optimized out>)
> at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> ---Type <return> to continue, or q <return> to quit---
> Thread 2 (process 28190):
> #0 0x0000003d05a0a899 in pthread_cond_wait@@GLIBC_2.3.2 ()
> from /lib64/libpthread.so.0
> #1 0x000000000046b7bc in sys_msg_dispatcher_func (
> unused=<value optimized out>) at beam/erl_threads.h:634
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
>
> Thread 1 (process 28189):
> #0 0x0000003d05a0d2cb in read () from /lib64/libpthread.so.0
> #1 0x0000000000523f6d in signal_dispatcher_thread_func (
> unused=<value optimized out>) at sys/unix/sys.c:2913
> #2 0x000000000057ca34 in thr_wrapper (vtwd=<value optimized out>)
> at common/ethread.c:475
> #3 0x0000003d05a06367 in start_thread () from /lib64/libpthread.so.0
> #4 0x0000003d04ed309d in clone () from /lib64/libc.so.6
> (gdb)
> (gdb)
> (gdb) quit
>
> Abridged output from "cat /proc/cpuinfo":
>
> processor : 0
> vendor_id : GenuineIntel
> cpu family : 6
> model : 26
> model name : Intel(R) Xeon(R) CPU L5520 @ 2.27GHz
> stepping : 5
> cpu MHz : 1596.000
> cache size : 8192 KB
> physical id : 1
> siblings : 8
> core id : 0
> cpu cores : 4
> [...]
>
>
> ________________________________________________________________
> erlang-bugs mailing list. See http://www.erlang.org/faq.html
> erlang-bugs (at) erlang.org
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: otp_src_R13B02-1_OTP-8297.patch
Type: text/x-patch
Size: 602 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20091118/2e200422/attachment.bin>
More information about the erlang-bugs
mailing list