Inter Log collector communication error - "Client authentication failed SSL verification error, err: 55"
Symptom
The following indicators are observed when inter-Log Collector (LC) communication fails due to certificate validation errors:
1. Log Indicators
- The
configd.logfile contains explicit failure messages regarding the ring connection:
admin@macs1-panorama(primary-active)> less mp-log configd.log
- The
ms.logfile reveals underlying SSL/Certificate validation errors, specifically identifying an Extended Key Usage (EKU) mismatch:
admin@macs1-panorama(primary-active)> less mp-log ms.log
Error: pan_sec_conn_cms_validation_impl(pan_sec_conn_server.c:445): [Secure conn verify result] Certificate verification failed due to Error 55 : Invalid Extended Key Usage Purpose
Error: pan_async_ssl_perform_enhanced_validation(cs_async_ssl.c:95): [Secure conn cms validation stage] Failed to validate the connection.
Error: pan_async_ssl_verify_conn(cs_async_ssl.c:146): Authorization of the incoming client failed.
2. Command Line Verification
Running the following operational command confirms the absence of active connections between the peers:
Server error : No active inter LC connections found.
3. Visual Indicators
The WebUI or runtime status may show a Disconnected state for inter-LC communication in the ring, correlating with the logs above.
Environment
- PAN-OS 9.1 and above
- Any Panorama when custom certificates are used for secure communication between log collectors.
Cause
The certificate used for inter Log collector communication should have both "Server Authentication" and "Client Authentication" attributes in "Extended Key Usage" section of the certificate.
Resolution
- To enable secure communication between log collectors either as dedicated log collector or in Panorama mixed mode Go to Panorama > Collector Group > General > Enable secure inter LC Communication
- To enable "Custom server Communication" and "Secure client communication" Go to Panorama >Managed Collectors and then select SSL/TLS Service Profile and Certificate Profile to enable secure server and client communication.
- Only use a certificate in these profiles which must have "Server Authentication" and "Client Authentication" key set in the "Extended key usage" section of the certificate as depicted in the below snapshot:
- Use correct certificate attributes with both "Server Authentication" and "Client Authentication" set in the "Extended Key Usage" section of certificate:
- By default, both attributes are added to the certificates generated on the PAN-OS devices.
- If you are using an external CA, make sure the certificate template in the CA server has both attributes before signing the Certificate Signing Request (CSR)
Additional Information
Configure Authentication with Custom Certificates Between Log Collectors
How Are SSL/TLS Connections Mutually Authenticated?