<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    Looking into kernel threads (both current CPU utilisation and
    consumed CPU time) in something like ps / htop should help.<br>
    <br>
    Cheers,<br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 20-12-2014 16:49, Guilherme Andrade
      wrote:<br>
    </div>
    <blockquote cite="mid:5495A8A5.7040103@gandrade.net" type="cite">
      <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
      Now that I look more carefully, it looks like your system is
      spending a whole lot of CPU time (~75%) in kernel space. Some
      system call (or calls) is taking away way too many resources.<br>
      <br>
      <br>
      <div class="moz-cite-prefix">On 20-12-2014 16:19, Guilherme
        Andrade wrote:<br>
      </div>
      <blockquote cite="mid:5495A19E.2020405@gandrade.net" type="cite">
        <meta content="text/html; charset=utf-8"
          http-equiv="Content-Type">
        Hello gongtao,<br>
        <br>
        Besides 'message_queue_len', I would also look into 'reductions'
        over time (also using process_info/2.)<br>
        <br>
        As for big message queues, if the CPU is not overloaded, in most
        cases it's due to single processes becoming a bottleneck,
        whether due to IO and/or heavy crunching; if they can be pooled
        or split into multiple ones, the better. However, you say that
        CPU is topping at 100%, so it's most likely that it's just too
        heavy.<br>
        <br>
        Long-running NIFs could also cause weird behaviour in the
        scheduler.<br>
        <br>
        This dirty (multiple) one-liner should give you some insight
        into reduction count:<br>
        <br>
        ------------------<br>
        rp([{-Reduc, Pid, case process_info(Pid, registered_name) of
        {registered_name,Name} -> Name; _ -> '_' end} || <br>
            {Reduc, Pid} <- <br>
            lists:foldl( <br>
                fun(Pid, L) when length(L) > 40 -> <br>
                        SL = lists:sublist(lists:keysort(1, L), 20), <br>
                        case process_info(Pid, reductions) of <br>
                            {reductions,Reduc} -> [{-Reduc, Pid} |
        SL]; <br>
                            undefined -> L <br>
                        end; <br>
                    (Pid, L) -> <br>
                        case process_info(Pid, reductions) of<br>
                            {reductions,Reduc} -> [{-Reduc, Pid} |
        L]; <br>
                            undefined -> L <br>
                        end <br>
                end, [], erlang:processes())]).<br>
        ------------------<br>
        <br>
        <br>
        Cheers,<br>
        <br>
        <div class="moz-cite-prefix">On 20-12-2014 12:03, gongtao wrote:<br>
        </div>
        <blockquote
          cite="mid:4afb7459.10962.14a67946226.Coremail.goto310@163.com"
          type="cite">
          <div
            style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial">
            <div>hello everyone:</div>
            <div>    I am a programmer from china.I have a problem with
              my game server.There is 8193 processes in my erlang
              node.But the cpu usage is nearly full,and the computer
              have 24 cpus.I looked some erlang processes info,find the
              message_queue_len of process is very big,more than 1000
              sometimes.The disk io and net io are normal,memory is
              normal.My erlang version is R15B03.I don't know why.</div>
            <div>    My english is poor.There are same informations from
              system.Hope you can understand.</div>
            <div>    Looking forward to your reply.</div>
            <div>    Thanks.</div>
            <div><br>
            </div>
            <div>
              <div>-bash-4.2$ sar -u 2 10</div>
              <div>Linux 2.6.32-279.el6.x86_64          12/20/2014    
                 _x86_64_        (24 CPU)</div>
              <div><br>
              </div>
              <div>06:33:37 PM     CPU     %user     %nice   %system  
                %iowait    %steal     %idle</div>
              <div>06:33:39 PM     all     11.21      0.00     75.04    
                 0.00      0.00     13.75</div>
              <div>06:33:41 PM     all     10.91      0.00     77.22    
                 0.00      0.00     11.86</div>
              <div>06:33:43 PM     all     12.08      0.00     75.96    
                 0.00      0.00     11.97</div>
              <div>06:33:45 PM     all     10.50      0.00     77.71    
                 0.00      0.00     11.79</div>
              <div>06:33:47 PM     all     11.09      0.00     76.75    
                 0.00      0.00     12.16</div>
              <div>06:33:49 PM     all     11.07      0.00     76.03    
                 0.00      0.00     12.90</div>
              <div>06:33:51 PM     all     12.07      0.00     75.27    
                 0.00      0.00     12.66</div>
              <div>06:33:53 PM     all     11.36      0.00     76.84    
                 0.00      0.00     11.80</div>
            </div>
            <div><br>
            </div>
            <div><br>
            </div>
            <div>
              <div>| CPU Utilisation
                -------------------------------------------------------------------------------------------------------------------------------------





                    |</div>
              <div>|---------------------------+-------------------------------------------------+





                                                                       
                                    |</div>
              <div>|CPU  User%  Sys% Wait% Idle|0          |25        
                |50          |75       100|                            
                                                                |</div>
              <div>|  1  14.1  75.9   0.0  
                9.9|UUUUUUUsssssssssssssssssssssssssssssssssssss  
                 >|                                                  
                                          |</div>
              <div>|  2   9.8  81.3   0.0  
                8.8|UUUUssssssssssssssssssssssssssssssssssssssss  
                 >|                                                  
                                          |</div>
              <div>|  3  11.6  76.9   0.0
                 11.6|UUUUUssssssssssssssssssssssssssssssssssssss    
                 >                                                  
                                          |</div>
              <div>|  4   7.6  82.3   0.0
                 10.1|UUUsssssssssssssssssssssssssssssssssssssssss >
                  |                                                    
                                        |</div>
              <div>|  5  14.2  74.1   0.0
                 11.7|UUUUUUUsssssssssssssssssssssssssssssssssssss  >
                 |                                                      
                                      |</div>
              <div>|  6   8.6  81.3   0.0
                 10.1|UUUUssssssssssssssssssssssssssssssssssssssss  >
                 |                                                      
                                      |</div>
              <div>|  7  11.6  75.9   0.0
                 12.6|UUUUUsssssssssssssssssssssssssssssssssssss      
                >                                                    
                                        |</div>
              <div>|  8   8.5  81.0   0.0
                 10.5|UUUUssssssssssssssssssssssssssssssssssssssss >
                  |                                                    
                                        |</div>
              <div>|  9  11.7  76.1   0.0
                 12.2|UUUUUssssssssssssssssssssssssssssssssssssss    
                 >                                                  
                                          |</div>
              <div>| 10   9.1  80.8   0.0
                 10.1|UUUUssssssssssssssssssssssssssssssssssssssss    
                >                                                    
                                        |</div>
              <div>| 11  10.2  76.5   0.0
                 13.3|UUUUUssssssssssssssssssssssssssssssssssssss  
                 > |                                                
                                            |</div>
              <div>| 12   6.6  81.6   0.0
                 11.7|UUUssssssssssssssssssssssssssssssssssssssss    
                 >                                                  
                                          |</div>
              <div>| 13  12.1  75.3   0.0
                 12.6|UUUUUUsssssssssssssssssssssssssssssssssssss    
                 >                                                  
                                          |</div>
              <div>| 14   9.0  80.6   0.0
                 10.4|UUUUssssssssssssssssssssssssssssssssssssssss    
                >                                                    
                                        |</div>
              <div>| 15  15.7  71.1   0.0
                 13.2|UUUUUUUsssssssssssssssssssssssssssssssssss    
                 >|                                                  
                                          |</div>
              <div>| 16   9.2  80.1   0.0
                 10.7|UUUUssssssssssssssssssssssssssssssssssssssss  >
                 |                                                      
                                      |</div>
              <div>| 17  13.6  74.2   0.0
                 12.1|UUUUUUsssssssssssssssssssssssssssssssssssss   >
                 |                                                      
                                      |</div>
              <div>| 18   7.2  82.1   0.0
                 10.8|UUUsssssssssssssssssssssssssssssssssssssssss >
                  |                                                    
                                        |</div>
              <div>| 19  16.5  69.5   0.0
                 14.0|UUUUUUUUssssssssssssssssssssssssssssssssss   >
                  |                                                    
                                        |</div>
              <div>| 20   7.3  81.2   0.0
                 11.5|UUUssssssssssssssssssssssssssssssssssssssss  
                 > |                                                
                                            |</div>
              <div>| 21  17.8  69.0   0.0
                 13.2|UUUUUUUUssssssssssssssssssssssssssssssssss    
                > |                                                  
                                          |</div>
              <div>| 22  11.4  77.2   0.0
                 11.4|UUUUUssssssssssssssssssssssssssssssssssssss >  
                 |                                                      
                                      |</div>
              <div>| 23  14.4  71.1   0.0
                 14.4|UUUUUUUsssssssssssssssssssssssssssssssssss   >
                  |                                                    
                                        |</div>
              <div>| 24   5.8  81.7   0.0
                 12.6|UUssssssssssssssssssssssssssssssssssssssss >  
                  |                                                    
                                        |</div>
              <div>|---------------------------+-------------------------------------------------+





                                                                       
                                    |</div>
              <div>|Avg  11.0  77.3   0.0
                 11.7|UUUUUssssssssssssssssssssssssssssssssssssss >  
                 |                                                      
                                      |</div>
              <div>|---------------------------+-------------------------------------------------+





                 </div>
            </div>
            <div><br>
            </div>
          </div>
          <br>
          <br>
          <span title="neteasefooter"><span id="netease_mail_footer"></span></span><br>
          <fieldset class="mimeAttachmentHeader"></fieldset>
          <br>
          <pre wrap="">_______________________________________________
erlang-questions mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
        </blockquote>
        <br>
        <pre class="moz-signature" cols="72">-- 
Guilherme

<a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://www.gandrade.net/">https://www.gandrade.net/</a>
PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A  D67A 9330 79B1 35CB 8191
</pre>
        <br>
        <fieldset class="mimeAttachmentHeader"></fieldset>
        <br>
        <pre wrap="">_______________________________________________
erlang-questions mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
      </blockquote>
      <br>
      <pre class="moz-signature" cols="72">-- 
Guilherme

<a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://www.gandrade.net/">https://www.gandrade.net/</a>
PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A  D67A 9330 79B1 35CB 8191
</pre>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
erlang-questions mailing list
<a class="moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>
<a class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
    </blockquote>
    <br>
    <pre class="moz-signature" cols="72">-- 
Guilherme

<a class="moz-txt-link-freetext" href="https://www.gandrade.net/">https://www.gandrade.net/</a>
PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A  D67A 9330 79B1 35CB 8191
</pre>
  </body>
</html>