Firewalls unable to fetch CDL certificate with error "Failed to load the JSON file supplied by the server"
13416
Created On 10/15/22 12:14 PM - Last Modified 03/07/23 20:59 PM
Symptom
- Palo Alto Strata firewalls are managed by Panorama with CDL (Cortex Data lake) (Formerly Logging service) License.
- The firewalls fail to fetch the certificate required for them to connect and send logs to CDL.
- The logging service status shows following failures.
admin@PA-fw> request logging-service-forwarding status Logging Service Certificate information: Info: Error sending CSR signing request to Panorama Not Valid after: 2022-10-21 00:00:21 Not Valid before: 2022-07-23 00:00:21 Status: failure Last fetched: Tue Oct 11 17:04:53 2022 Logging Service Customer file information: Info: Failed to validate server certificate for endpoint api.paloaltonetworks.com Status: failure - The certificate fetch job fails.
admin@PA-fw> request logging-service-forwarding certificate fetch Successfully scheduled logging service certificate fetch job with a job id of 34292 admin@PA-fw> show jobs id 34292 Enqueued Dequeued ID Type Status Result Completed ------------------------------------------------------------------------------------------------------------------------------ 2022/10/07 14:45:59 14:45:59 34292 LCaaS-certificate-fetch FIN FAIL 100 % - The ms.logs on the firewall show following logs.
2022-10-10 00:04:48.239 -0700 Error: pan_valid_lcaas_cert_present(pan_lcaas_cert_ops.c:663): Either certificate or key or both are missing 2022-10-10 00:04:48.239 -0700 LCAAS_CERT_RENEWAL scheduled 2022-10-10 00:04:48.239 -0700 Succesfully scheduled logging service certificate fetch job with a job id of 36069 2022-10-10 00:04:48.239 -0700 LCAAS_CERT_RENEWAL cert thread start 2022-10-10 00:04:48.239 -0700 LCAAS_CERT_RENEWAL fetch cert start Generating RSA private key, 2048 bit long modulus ........................................+++ .....+++ e is 65537 (0x10001) 2022-10-10 00:04:49.355 -0700 Succesfully stored logging service priv_key (169949001_lcaas_cert_priv_key) in cryptod 2022-10-10 00:04:50.268 -0700 Succesfully sent logging service CSR signing request to panorama 2022-10-10 00:04:50.268 -0700 Response for cert fetch from CSP: <response status="success"><result><lcaas-cert>None</lcaas-cert><err-msg>Failed to load the JSON file suppl ied by the server. </err-msg><debug-msg/> <status>Failure</status> </result></response> 2022-10-10 00:04:50.268 -0700 Error: pan_lcaas_handle_cert_push(pan_lcaas_cert_ops.c:1180): Error response: Failed to load the JSON file supplied by the server. 2022-10-10 00:04:50.270 -0700 Error: _send_csr_signing_request_plugin(pan_lcaas_cert_ops.c:585): Failed to either write certificate file to disk 2022-10-10 00:04:50.270 -0700 Error: pan_lcaas_fetch_cert(pan_lcaas_cert_ops.c:908): Error sending CSR signing request to Panoram
Environment
- Palo Alto Strata firewalls managed by Panorama.
- PanOS 10.0 or below
Cause
- This is caused by the disconnection of Cloud service plugin on the panorama with the CDL backend.
- The Cloud service plugin on the panorama handles the connection between panorama and the CDL backend.
- The plugin connection may break when the licenses are renewed and OTP verification needs to be done again.
Resolution
- Perform the OTP verification on the panorama for Cloud service plugin.
- If any errors with the process, Use this article for help.