How to the interpret output of "show system resources" for multicore CPU

How to the interpret output of "show system resources" for multicore CPU

12985
Created On 04/13/19 04:52 AM - Last Modified 04/18/19 20:26 PM


Objective
This article explains the interpretation of "show system resources" command for firewalls having multi core CPU for MP

Environment
Applicable for all PA Firewalls with more than one CPU core for MP

Procedure
Most of the Palo Alto Platforms have multiple core CPUs. Some platforms have dedicated processors for MP and DP, while some use Single Processor for both MP and DP. 

For example, 
 
PlatformMP ProcessorsMP Cores
PA-200Single Shared Dual Core Processor1
PA-220Single Shared Quad Core Processor2
PA-800Single Shared 8 Cores Processor3
PA-3000Dedicated Dual Core Processor2
PA-3200Dedicated Quad Core Processor4
PA-5000Dedicated Quad Core Processor4
PA-5200Dedicated Multi Core Processor16-24

The "show system resources" command can be used to monitor the MP CPU statistics of the Processor which is handling MP functionality. This command is equivalent of Linux Command "top"

However since this processor can be multiple core processor (some cores can also be used for DP depending on platform), the output lists usage for all cores combined. 

You can press "1" after running "show system resources follow" to toggle view to show separate states:

Example from a PA-850 device:
 
> show system resources follow

top - 21:24:45 up 41 days,  4:07,  1 user,  load average: 5.38, 5.16, 5.09
Tasks: 155 total,   6 running, 149 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4119472k total,  3328804k used,   790668k free,   303532k buffers
Swap:  2008088k total,       60k used,  2008028k free,   896576k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                                               
 3137 root      20   0 74552  33m 6360 R 100.0  0.8  59281:29 pan_task                                                                                                                                                                                              
 3138 root      20   0 49716 8864 6292 R 100.0  0.2  59281:57 pan_task                                                                                                                                                                                              
 3139 root      20   0 49716 8856 6284 R 100.0  0.2  59282:01 pan_task                                                                                                                                                                                              
 3140 root      20   0 49716 8772 6192 R 100.0  0.2  59281:39 pan_task                                                                                                                                                                                              
 3141 root      20   0 49716 8808 6236 R 100.0  0.2  59281:43 pan_task                                                                                                                                                                                              
17662 admin     20   0  5248 1568 1108 R  0.7  0.0   0:00.08 top                                                                                                                                                                                                    
 3145 root      30  10 65188  13m 5780 S  0.3  0.3  38:40.36 python                                                                                                                                                                                                 
 3352 root      20   0  478m  25m  15m S  0.3  0.6  10:02.50 ikemgr                                                                                                                                                                                                 
    1 root      20   0  3212  852  688 S  0.0  0.0   0:27.34 init                                                                                                                                                                                                   
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.11 kthreadd
 
  • You can see above this box has 8 CPUs, 5 of which (1-5) are for DP. The DP process is pan_task which is supposed to be running at all times hence it has no idle CPU cycles.
  • In above you will notice individually each pan_task (process for DP) are showing 100% CPU. This is design behavior of TOP Command in IRIX Mode where It is possible for the % CPU column to display values that total greater than 100%
  • Solaris mode divides the % CPU for each process by the number of CPUs in the system so that the total will be 100%.

To Toggle IRIX Mode off press Shift+I:

top - 21:36:17 up 41 days,  4:18,  1 user,  load average: 5.00, 5.04, 5.06
Tasks: 155 total,   6 running, 149 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.7%sy,  0.0%ni, 99.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  0.0%us,  0.7%sy,  0.7%ni, 98.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4119472k total,  3329324k used,   790148k free,   303612k buffers
Swap:  2008088k total,       60k used,  2008028k free,   896872k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                                               
 3137 root      20   0 74552  33m 6360 R 12.5  0.8  59293:01 pan_task                                                                                                                                                                                               
 3138 root      20   0 49716 8864 6292 R 12.5  0.2  59293:29 pan_task                                                                                                                                                                                               
 3139 root      20   0 49716 8856 6284 R 12.5  0.2  59293:33 pan_task                                                                                                                                                                                               
 3140 root      20   0 49716 8772 6192 R 12.5  0.2  59293:11 pan_task                                                                                                                                                                                               
 3141 root      20   0 49716 8808 6236 R 12.5  0.2  59293:15 pan_task                                                                                                                                                                                               
 2789 root      20   0 40220 5372 3876 S  0.1  0.1 260:22.65 ehmon                                                                                                                                                                                                  
 2760 root      30  10 65380  14m 5816 S  0.0  0.4 144:06.87 python                                                                                                                                                                                                 
12901 root      20   0     0    0    0 S  0.0  0.0   0:11.08 kworker/7:2                                                                                                                                                                                            
19622 admin     20   0  5248 1560 1108 R  0.0  0.0   0:00.09 top

Additionally note that average CPU usage for this box will also consider DP Cores so your monitoring tools will show higher MP Usage.


Example from a PA-5060 device:
 
> show system resources follow

 top - 21:23:26 up 4 days,  1:57,  1 user,  load average: 0.00, 0.00, 0.00
Tasks: 131 total,   1 running, 130 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.3%us,  0.0%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4053832k total,  2953796k used,  1100036k free,   163596k buffers
Swap:  2007992k total,     1080k used,  2006912k free,  1298156k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                                               
 2402 root       0 -20 52960  16m 4440 S  0.3  0.4   5:31.90 masterd_apps                                                                                                                                                                                           
 2421 root      15  -5 21680 4408 2508 S  0.3  0.1  18:59.06 sysd                                                                                                                                                                                                   
 3106 nobody    20   0  156m  47m  10m S  0.3  1.2  21:01.78 appweb3                                                                                                                                                                                                
    1 root      20   0  2084  696  576 S  0.0  0.0   0:01.88 init

In this platform all cores are dedicated for MP, so you do not see any pan_task. Rest of the logic remains same.  


Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000PLZZCA4&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail

Attachments