How to the interpret output of "show system resources" for multicore CPU
46673
Created On 04/13/19 04:52 AM - Last Modified 04/18/19 20:26 PM
Objective
This article explains the interpretation of "show system resources" command for firewalls having multi core CPU for MP
Environment
Applicable for all PA Firewalls with more than one CPU core for MP
Procedure
Most of the Palo Alto Platforms have multiple core CPUs. Some platforms have dedicated processors for MP and DP, while some use Single Processor for both MP and DP.
For example,
| Platform | MP Processors | MP Cores |
|---|---|---|
| PA-200 | Single Shared Dual Core Processor | 1 |
| PA-220 | Single Shared Quad Core Processor | 2 |
| PA-800 | Single Shared 8 Cores Processor | 3 |
| PA-3000 | Dedicated Dual Core Processor | 2 |
| PA-3200 | Dedicated Quad Core Processor | 4 |
| PA-5000 | Dedicated Quad Core Processor | 4 |
| PA-5200 | Dedicated Multi Core Processor | 16-24 |
The "show system resources" command can be used to monitor the MP CPU statistics of the Processor which is handling MP functionality. This command is equivalent of Linux Command "top"
However since this processor can be multiple core processor (some cores can also be used for DP depending on platform), the output lists usage for all cores combined.
You can press "1" after running "show system resources follow" to toggle view to show separate states:
Example from a PA-850 device:
> show system resources follow
top - 21:24:45 up 41 days, 4:07, 1 user, load average: 5.38, 5.16, 5.09
Tasks: 155 total, 6 running, 149 sleeping, 0 stopped, 0 zombie
Cpu0 : 0.3%us, 0.0%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu4 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu5 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu6 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu7 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 4119472k total, 3328804k used, 790668k free, 303532k buffers
Swap: 2008088k total, 60k used, 2008028k free, 896576k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3137 root 20 0 74552 33m 6360 R 100.0 0.8 59281:29 pan_task
3138 root 20 0 49716 8864 6292 R 100.0 0.2 59281:57 pan_task
3139 root 20 0 49716 8856 6284 R 100.0 0.2 59282:01 pan_task
3140 root 20 0 49716 8772 6192 R 100.0 0.2 59281:39 pan_task
3141 root 20 0 49716 8808 6236 R 100.0 0.2 59281:43 pan_task
17662 admin 20 0 5248 1568 1108 R 0.7 0.0 0:00.08 top
3145 root 30 10 65188 13m 5780 S 0.3 0.3 38:40.36 python
3352 root 20 0 478m 25m 15m S 0.3 0.6 10:02.50 ikemgr
1 root 20 0 3212 852 688 S 0.0 0.0 0:27.34 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.11 kthreadd
- You can see above this box has 8 CPUs, 5 of which (1-5) are for DP. The DP process is pan_task which is supposed to be running at all times hence it has no idle CPU cycles.
- In above you will notice individually each pan_task (process for DP) are showing 100% CPU. This is design behavior of TOP Command in IRIX Mode where It is possible for the % CPU column to display values that total greater than 100%
- Solaris mode divides the % CPU for each process by the number of CPUs in the system so that the total will be 100%.
To Toggle IRIX Mode off press Shift+I:
top - 21:36:17 up 41 days, 4:18, 1 user, load average: 5.00, 5.04, 5.06 Tasks: 155 total, 6 running, 149 sleeping, 0 stopped, 0 zombie Cpu0 : 0.0%us, 0.7%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu4 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu5 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu6 : 0.0%us, 0.7%sy, 0.7%ni, 98.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu7 : 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 4119472k total, 3329324k used, 790148k free, 303612k buffers Swap: 2008088k total, 60k used, 2008028k free, 896872k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3137 root 20 0 74552 33m 6360 R 12.5 0.8 59293:01 pan_task 3138 root 20 0 49716 8864 6292 R 12.5 0.2 59293:29 pan_task 3139 root 20 0 49716 8856 6284 R 12.5 0.2 59293:33 pan_task 3140 root 20 0 49716 8772 6192 R 12.5 0.2 59293:11 pan_task 3141 root 20 0 49716 8808 6236 R 12.5 0.2 59293:15 pan_task 2789 root 20 0 40220 5372 3876 S 0.1 0.1 260:22.65 ehmon 2760 root 30 10 65380 14m 5816 S 0.0 0.4 144:06.87 python 12901 root 20 0 0 0 0 S 0.0 0.0 0:11.08 kworker/7:2 19622 admin 20 0 5248 1560 1108 R 0.0 0.0 0:00.09 top
Additionally note that average CPU usage for this box will also consider DP Cores so your monitoring tools will show higher MP Usage.
Example from a PA-5060 device:
> show system resources follow
top - 21:23:26 up 4 days, 1:57, 1 user, load average: 0.00, 0.00, 0.00
Tasks: 131 total, 1 running, 130 sleeping, 0 stopped, 0 zombie
Cpu0 : 0.3%us, 0.0%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 0.3%us, 0.0%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 4053832k total, 2953796k used, 1100036k free, 163596k buffers
Swap: 2007992k total, 1080k used, 2006912k free, 1298156k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2402 root 0 -20 52960 16m 4440 S 0.3 0.4 5:31.90 masterd_apps
2421 root 15 -5 21680 4408 2508 S 0.3 0.1 18:59.06 sysd
3106 nobody 20 0 156m 47m 10m S 0.3 1.2 21:01.78 appweb3
1 root 20 0 2084 696 576 S 0.0 0.0 0:01.88 init
In this platform all cores are dedicated for MP, so you do not see any pan_task. Rest of the logic remains same.