[erlang-questions] Timeout Erlang GenServer Crash Loop

Michael Truog <>
Fri Oct 12 07:18:27 CEST 2012


Well a common problem is to have the process also blocked on its own synchronous call, so that can keep the CPU usage low, since it is spending time mostly idle waiting for 1 or more responses from some other processes.  The best way I have seen to deal with this type of timeout problem is to always pass the timeouts in the message like this:
gen_server:call(<process>, {<message>, Timeout - DELTA}, Timeout)
Where DELTA can be 100 milliseconds.  Then the (Timeout-DELTA) value the handle_call sees can be used for any internally synchronous calls.  However, then the problem becomes understanding what the cumulative delay might be, if there are multiple synchronous calls used within the process.  Ideally, the process is kept simpler, so it doesn't need to try and track many synchronous calls.

I am not entirely sure if this is your problem, since it could be latency due to function calls too, if function calls are blocking schedulers or something strange, code loading locking schedulers.  Usually those issues aren't as common a concern though.

On 10/11/2012 10:05 PM, Code Box wrote:
> Will it not relate to any CPU Stats of my host and also any memory stats of my host that the process is overloaded ? I see CPU % usage as just 50% ?
>
> On Thu, Oct 11, 2012 at 9:14 PM, Michael Truog < <mailto:>> wrote:
>
>     On 10/11/2012 09:03 PM, Code Box wrote:
>>     ** Reason for termination ==
>>     ** {timeout,{gen_server,call,[thetime,gettime]}}
>>
>>     =CRASH REPORT==== 2012-10-09 05:37:04 UTC ===
>>       crasher:
>>         initial call: process_listener:-init/1-fun-2-/0
>>         pid: <0.12376.513>
>>         registered_name: []
>>         exception exit: {timeout,{gen_server,call,[thetime,gettime]}}
>>           in function  gen_server:terminate/6
>>         ancestors: [incoming_req_processor,incoming_sup,top_process_sup,
>>                       <0.52.0>]
>>         messages: []
>>         links: []
>>         dictionary: [{random_seed,{23375,22820,17046}}]
>>         trap_exit: true
>>         status: running
>>         heap_size: 6765
>>         stack_size: 24
>>         reductions: 1646842
>>       neighbours:
>>
>>     I am seeing a lot of these messages in my Crash Reports. Once this reaches this it goes into this crash loop for quite a while. I am not sure how to debug this error. These timeouts are really annoying. Can some one help me understand the root cause of this?
>>
>>     Why does my genserver calls are facing timeouts ? Is it that my erlang VM is slow if so why ? How can i debug this issue to get to the root cause of it ? 
>>
>     If you look at gen_server:call/2 at http://www.erlang.org/doc/man/gen_server.html
>     it shows the default Timeout is 5000 milliseconds (5 seconds).  Your gen_server process must have been processing for longer than 5 seconds while a gen_server:call/2 message was waiting in the process message queue, to cause the timeout exception.  So, it isn't the Erlang VM being slow, it is just an Erlang process that is overloaded (i.e., the "thetime" registered process).
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121011/b1e60163/attachment.html>


More information about the erlang-questions mailing list