Firewalls disconnected from Panorama with error message "SC3: verify error:27:certificate not trusted (depth:0)" in ms.log
11155
Created On 09/29/23 05:41 AM - Last Modified 03/18/25 06:01 AM
Symptom
- Multiple Firewalls suddenly disconnected from the primary-active Panorama and/or commit push to device failed with the Error message: "Panorama connectivity check failed for <IP>. Reason: SSL handshake failed, reverting configuration".
- The firewalls are connected to the secondary Panoramas correctly.
- Ping between Firewalls and Panorama works but the SSL is failing.
- ms.log (less mp-log ms.log) on Panorama shows Error 'Unknown SNI'.
- ms.log/configd.log (less mp-log configd.log) on FW shows Error 'SC3: verify error:27:certificate not trusted (depth:0)'
- When checking the SDB value using "show system state filter cfg.ms.*" on both Panoramas and Firewalls, CA value on primary-active does not match with the CA on the secondary-passive Panorama and on Firewalls.
Environment
- Panorama managed Firewalls
- PAN-OS 10.1 or higher
- Panorama configured in HA (High Availability)
Cause
CA certificate is changed on the Primary-active Panorama (SC3.CA change) which does not match the certificate on the Firewall.
Resolution
Perform the following steps to resolve the issue:
- Suspend Primary-Active Panorama (A).
- Secondary Panorama becomes active Secondary-Active (B).
- Then change the priority of (B) to Primary and (A) to Secondary and commit (ensure pre-emption is enabled).
- This will sync the CA certificate from (B) to (A) and will matches with the CA on Firewalls.
- Make Panorama (A) functional and check the SDB value to confirm the match.
- After this change, the firewalls should be connected to both Panoramas automatically.
Additional Information
Changes to the SC3.CA certificate on Panorama could occur due to:
- A reset of SC3 on Panorama.
- Cases where a new Panorama is introduced via RMA.
- Changes to SC3.CA on Panorama during periods of high CPU usage. If a connection flap occurs between the FW and Panorama, the SDB fetch may fail to read SC3.CA.