[erlang-bugs] R11B-5 to R12B-3 forks beams eating 100% CPU

[Lev Walkin]

correction:

	Disabling yaws application DOES seem to have effect on
	the rate of runaway beam creation.


Lev Walkin wrote:
> SYMPTOMS:
> 
> 	A running, unstressed erlang system appears to spontaneously
> 	fork off BEAM processes (beam) several times an hour,
> 	each eating 100% CPU. (If there are more than one at a time,
> 	they split the CPU accordingly).
> 
> INVESTIGATION:
> 
> 	The BEAM processes forked appear to be truly separate processes.
> 	Killing them (-9) does not seem to harm the main VM, which
> 	continues to execute the necessary set of applications
> 	(kernel, stdlib, sasl, ssl, yaws, plus two inhouse ones).
> 
> 	Disabling yaws application does not seem to have any effect on
> 	the rate of runaway beam creation.
> 
> 	ktrace shows a total absence of any system calls in the runaway
> 	beam process, suggesting an infinite loop. Killing the process
> 	results in a single ktrace event:
> 
> 	[root@REDACTED ~]# kdump
> 	 97445 beam     PSIG  SIGKILL SIG_DFL
> 	[root@REDACTED ~]#
> 
> 	The R12B-3 has the following fix:
> 
> 
> 	    OTP-7289  On Mac OS 10.5 (Leopard), sending to socket
> 	              which the other end closes could cause the
> 	              emulator to consume 100% CPU time.
> 	              (Thanks to Matthias Radestock.)
> 
> 	Since Mac OS is sufficiently similar to FreeBSD, I had hoped
> 	the OTP-7289 has direct relation to my runaway beam problem.
> 	Alas, R12B-3 behaves exactly the way R11B-5 and the latter
> 	versions of Erlang VM behave, that is, runaway beams
> 	are continuing to appear on a regular basis.
> 
> WORKAROUND USED:
> 
> 	Created an external process which periodically detects
> 	and kills the runaway beams. Ugly.
> 
> CONFIGURATION:
> 
> 	Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] 
> [async-threads:0] [hipe] [kernel-poll:false]
> 
> 	FreeBSD host 6.3-RELEASE FreeBSD 6.3-RELEASE #0: Wed Jan 16 01:43:02 
> UTC 2008     root@REDACTED:/usr/obj/usr/src/sys/SMP  amd64
> 
> 	However, the system was running all versions of erlang VM
> 	between R11B-5 and R12B-3 with the same results, so
> 	it is not version specific.
> 
> 
> Please suggest further steps to debug or eliminate the problem.
> 
>