PA-5450 has unexpected high availability failover due to path_monitor HB failures

• Device with large sessions can trigger an internal path monitoring failure all of a sudden resulting in HA failover.
• mp-monitor.log (less mp-log mp-monitor.log) show core1 (DP0) to be under a high load.  All other DP cores are normal.

CPU load (%) during last 60 seconds:
   core 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
       *  5 4 4 4 3 4 3 3 3 4 4 4 3 3 3
       *  6 5 4 4 4 4 4 4 4 4 4 4 4 4 4
       *  9 7 6 6 6 6 6 6 6 6 6 6 6 6 6
       *  12 9 8 8 8 8 8 8 8 8 8 8 8 8 8
       *  13 10 9 9 9 9 9 9 9 9 8 8 9 9 9
       *  11 9 8 8 7 8 8 8 8 8 8 8 8 8 8
       *  9 7 6 6 6 6 6 6 6 6 6 6 6 6 6
       * 10 7 6 6 7 7 7 7 7 6 7 7 7 7 6
       * 12 9 8 8 8 8 8 8 8 8 8 8 8 8 8
       * 24 9 8 8 8 8 8 8 8 8 8 8 8 8 8
       * 70 9 8 9 9 9 9 9 8 9 9 9 8 9 9
       * 100 8 8 8 8 8 8 8 8 8 8 8 8 8 8
       * 100 8 8 8 8 8 8 8 8 8 8 8 8 8 8
       * 100 9 9 9 9 9 9 9 9 9 9 9 9 9 9
       * 100 13 12 12 12 13 12 13 12 13 12 12 12 12 13
       * 100 10 10 10 10 11 10 11 10 11 10 10 11 10 10
       * 100 10 10 10 10 10 10 10 10 10 10 10 10 10 10
       * 100 10 10 10 10 10 10 10 10 10 10 10 10 10 10 

• Firewall failover with the following errors in system log (show log system).   When failover occurs, it affects all DPC and NC cards.

 HA Slot 4 down; failure
 HA Slot 3 down; failure
 HA Slot 2 down; failure
 HA Slot 1 down; failure
 Chassis Master Alarm: HA-event
 slot3dp0-path_monitor HB failures seen, triggering HA DP down
 internal packet path monitoring failure, rastarting slot 5
 link-monitor-down

• Global counters (show counter global) display missing poll interval. Packets missing between Poll1 vs Poll2

Poll 1
flow_path_monitor_rx_from_mp 980489 27 info flow mgmt Path monitor packet received from MP
flow_path_monitor_tx_to_mp 980419 27 info flow mgmt Path monitor packet sent to MP
flow_path_monitor_tx_to_fe 439752 9 info flow mgmt Path monitor packet sent to FE
flow_path_monitor_rx_from_fe 439682 9 info flow mgmt Path monitor packet received from FE
Poll 2:
flow_path_monitor_rx_from_mp 980559 33 info flow mgmt Path monitor packet received from MP + 70
flow_path_monitor_tx_to_mp 980479 33 info flow mgmt Path monitor packet sent to MP + 60
flow_path_monitor_tx_to_fe 439782 11 info flow mgmt Path monitor packet sent to FE + 30
flow_path_monitor_rx_from_fe 439702 11 info flow mgmt Path monitor packet received from FE + 20

• Palo Alto 5400 Series firewall
• PANOS 10.1.6 or below.
• PANOS 10.2.2 or below.

1. The issue is fixed in  PAN-OS 10.2.3, 10.1.7, 10.1.6-h4 under PAN-192551
2. Upgrade to the latest Preferred release to fix the issue.

PA-5450 Front Panel