High Dataplane utilization observed on a PA 3000 series firewalls with QoS enabled.
15622
Created On 01/10/20 08:01 AM - Last Modified 11/13/20 04:55 AM
Symptom
- Packet buffer utilization would be normal.
- Packet descriptors would be running high.
- Global Counters display High values observed for counters "pkt_recv ", "flow_qos_pkt_enque", flow_qos_pkt_deque as seen below.
----------------------------------------------------Counters from PA 3050---------------------------------------------------------------------------
pkt_recv 747016 183948 info packet pktproc Packets received
flow_qos_pkt_enque 703327 173190 info flow qos Packet enqueued to QoS module
flow_qos_pkt_deque 703343 173194 info flow qos Packet dequeued from QoS module
--------------------------------------------------------------------------------------------------------------------------------------------------------------
pkt_recv 747016 183948 info packet pktproc Packets received
flow_qos_pkt_enque 703327 173190 info flow qos Packet enqueued to QoS module
flow_qos_pkt_deque 703343 173194 info flow qos Packet dequeued from QoS module
--------------------------------------------------------------------------------------------------------------------------------------------------------------
- With QoS disabled, one can observe a drop in packet descriptor utilization and a significant drop in "pkt_recv" counter and there would be no counters for "flow_qos_pkt_enque" and "flow_qos_pkt_deque". Thereby, you would see a drop in data-plane utilization.
------------------------------------------------------Counters from PA 3050-------------------------------------------------------------------------
pkt_recv 46516 33634 info packet pktproc Packets received
pkt_recv 24067 34778 info packet pktproc Packets received
----------------------------------------------------------------------------------------------------------------------------------------------------------------
pkt_recv 46516 33634 info packet pktproc Packets received
pkt_recv 24067 34778 info packet pktproc Packets received
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Environment
- Any PAN-OS
- Palo Alto 3000 Series Firewall.
- QoS Enabled.
Cause
- On a PA-3000 series devices, QoS is done in software (on CPU).
- With QoS enabled, all the traffic ingressing the firewall will be hitting the Dataplane CPU (including the offloaded traffic) in order for the firewall to take the decision as to which traffic needs to be QoS throttled and this will increase the load on the DP.
- With QoS disabled some of the traffic will get offloaded thereby reducing the load on the DP.
Resolution
- Disable QoS
- Use the Platforms which support QoS in hardware (For 3200 Series Firewall) in this case.