High Availability pair randomly not in sync after Commit push from Panorama
63048
Created On 03/31/20 19:11 PM - Last Modified 04/22/24 20:17 PM
Symptom
- HA A/P is going out of sync randomly after a Panorama push
- HA A/P appears in sync at Panorama
- After running a config comparison there are no differences that would qualify it to be out of sync.
- No commit errors or similar situation reported
- The issue is resolved temporarily by performing manual sync from the Primary/Active.
- Both Active and Passive are reporting next alerts:
Active 2020/02/24 22:34:49 critical ha config- 0 HA Group 5: Running configuration not synchronized after failure Passive 2020/02/24 22:34:49 critical ha config- 0 HA Group 5: Running configuration not synchronized after failure
- The previous message is generated when "Commit All" is done on both of the HA firewalls in the pair, and "Merge with Device Candidate Config" is set on Panorama.
Environment
- Any Panorama managing Palo Alto Firewalls
- High Availability Configured.
- PAN-OS 7.1 and above.
- Commit all and Push from Panorama with "merge with device candidate config" is set to yes or "force template values" box checked
Cause
- If one of the HA devices finishes the Commit job faster than the HA peer and local config gets changed due to this commit, a device will try to initiate HA sync job to the peer. However, the peer is still doing the commit job so HA sync job will fail.
- When the "commit All" is in progress on this box, ha sync is going to fail to show next error:
- NOTE: Please review the configd.log/ms.log files on Panorama and the ha.log/ms.log files on the firewall to obtain a comprehensive sequence of events.
"2019-08-06 11:58:29.192 -0400 Commit jobs in queue on this box, please try again later
Resolution
- Uncheck "Merge with device candidate config" when pushing configuration to an HA pair firewall
- Locally commit the local changes to the firewall.