Packet Drops and Slow Performance after upgrading from PAN-OS 10.2.7 to 11.2

• Performance issues due to high-risk processing overhead.
• Resource exhaustion problems on platforms where on-chip packet descriptors are a limited hardware resource
• In particular:
  • High on-chip buffer utilization:

    <show running resource-monitor>
    resource 15m :packet descriptor (on-chip) (maximum) :packet descriptor (on-chip) (maximum):
    resource 15m :packet descriptor (on-chip) (maximum) : 19  28  21  23 100 100 100 100 100  15  19  16  12 100  19

  • Packet Buffer Protection actively working, which can be monitored by the following counters 

    flow_dos_pbp_drop
    

• Which may potentially lead to control plain instability (packet loss and routing protocols flapping)

• Palo Alto Networks Firewall: PA-220, PA-800, PA-3200, PA-5200, PA-7000
• PAN-OS: 10.2.8, 11.1.x, 11.2.0, 12.1.0, 13.1.0 
• Updating from PAN-OS 10.2.7 to 11.2
• TCP fast retransmission 

• Transmission Control Protocol (TCP) fast retransmission increases the rate of packets in flight to optimize network throughput in accordance with RFC 5681/6675.
• The accelerated packet transmission increases performance demand and consumes on-chip packet descriptors rapidly.
• Fixed-size on-chip packet descriptor pools are a limited hardware constraint inherent to the architecture of Gen 3 platforms.
• High processing workloads, such as Secure Sockets Layer (SSL) decryption, amplify descriptor consumption and can lead to hardware resource exhaustion.
• The behavior stems from a physical platform capacity limitation rather than a software defect or bug.

1. Log into the firewall Command Line Interface (CLI).
2. Execute the platform-specific operational runtime command to disable Transmission Control Protocol (TCP) fast retransmission and stabilize the packet descriptor resource pool:

debug dataplane set ssl-decrypt disable-new-fast-retransmit yes

3. Monitor system resources to verify that packet buffer protection (PBP) drops decrease and on-chip descriptor exhaustion is successfully mitigated.

Note: This runtime setting remains persistent across device reboots and software upgrades.