Simultaneous HA Firewall Failure and Dataplane Restart Loop Following System Reboot

Simultaneous HA Firewall Failure and Dataplane Restart Loop Following System Reboot

334
Created On 10/13/26 14:58 PM - Last Modified 10/13/26 16:35 PM


Symptom


  • Both firewalls in an Active/Passive or Active/Active HA cluster transition to a "Non-Functional" or "HA not enabled" state simultaneously.

  • Traffic outage occurs as neither unit can successfully retain active dataplane operations.

  • System logs (show log system) indicate repeated dataplane exits and HA failovers triggered by path monitoring heartbeat failures:

    critical general general 0 Internal packet path monitoring failure, restarting dataplane
    critical ha dataplane-down 0 HA Group 1: Dataplane is down: path monitor failure
    high general general 0 9: dp0-path_monitor HB failures seen, triggering HA DP down
    critical general general 0 dp0-path_monitor: Exited 1 times, must be manually recovered.
    
  • System logs reveal NMI Watchdog interrupts across CPU cores followed by dataplane connection timeouts in sysd.log:

    *** NMI Watchdog interrupt on Core 0x00 ***
    ...
    DP connection timed out x32
    
  • In chasd.log, the CPLD early dataplane down callback reports a null reason:

    Error: chasd_cpld_early_dp_down_cb(chasd_power.c:159): chasd_cpld_early_dp_down_cb reason (null)
    
  • Searching system logs for CPLD events (show log system | grep CPLD) returns no records, confirming that the hardware power-cycle recovery mechanism failed to trigger automatically.



Environment


  • Hardware Platforms: PA-3200 Series (PA-3220, PA-3250, PA-3260)

  • Operating System: PAN-OS 11.1.x



Cause


The issue is caused by a sequence of software regressions involving internal platform diagnostic and execution scripts.

Because the CPLD power cycle failed to execute during a path monitoring failure, the dataplane entered a continuous reboot/exit loop rather than performing a clean hardware reset to recover internal packet paths.



Resolution


This issue has been permanently resolved by ensuring CPLD power cycles are exclusively and correctly invoked upon internal path monitoring failures.

Upgrade the firewall to one of the following fixed release versions (or later):

  • PAN-OS 11.1.6-h38

  • PAN-OS 11.1.10-h32

  • PAN-OS 11.1.13-h11

  • PAN-OS 11.1.16-h1

  • PAN-OS 11.1.17



Additional Information


Workaround: If firewalls are currently trapped in this state, perform a full physical power cycle (hard reboot) on both units to clear the internal path lockup and restore normal HA operations.



Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA1hj0000000Ma1CAE&lang=en_US&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail