How to troubleshoot User-ID agent connection failure due to certificate verification failure?

• Troubleshoot User-ID agent connection failures due to certificate validation errors between a Palo Alto Networks Next-Generation Firewall (NGFW) and a User-ID agent.
• Following errors are observed in distributor.log where the User-ID agent uses a non-default certificate and the NGFW uses a default certificate.

> less mp-log distributor.log
...
Error:  pan_distributor_agent_verify_cert_cb(pan_distributor_agent.c:1836): X509_verify_cert returned error 20, error = 'unable to get local issuer certificate'
[distributord] Returning FAILURE from pan_user_id_uia_verify_cert_cb
Error:  pan_dcom_ssl_connect(pan_dcom_ssl.c:317): conn MY_USERID_AGENT: SSL_connect return -1

• Following errors are observed in distributor.log where the User-ID agent uses a default certificate and the NGFW uses a non-default certificate.

> less mp-log distributor.log
...
Error:  pan_distributor_agent_verify_cert_cb(pan_distributor_agent.c:1836): X509_verify_cert returned error 18, error = 'self signed certificate'
[distributord] Returning FAILURE from pan_user_id_uia_verify_cert_cb
Error:  pan_dcom_ssl_connect(pan_dcom_ssl.c:317): conn MY_USERID_AGENT: SSL_connect return -1

• Palo Alto Firewalls
• PAN-OS 11.1 and above
• User-ID Agent 11.0.2 and above
• Certificate Profile

This article presents three different ways to resolve the issue, arranged from the simplest procedure.

1. Use the Default Palo Alto Certificate embedded into the User-ID agent and the firewall
   1. In the User-ID agent, delete the configured server certificate by navigating to Server Certificate > Delete.
   2. Click Save and then, Commit.

1. 3. In the firewall, delete the configured Certificate profile referenced in User-ID Certification Profile.

GUI: Device > User Identification > Connection Security > User-ID Connection Security > User-ID Certification Profile > None

1. 4. Save the changes, click on OK and then, Commit.

2. Use Public Certificate Authority signed certificate to authenticate the User-ID agent's Server Certificate.
   1. Generate Certificate Signing Request Use "Manually Generate a Certificate Signing Request (CSR) using OpenSSL" for Guidance.
   2. Submit the CSR to your chosen Public CA
   3. Follow the steps below from 3.g as illustrated to install the user-ID Agents Server certificate

3. Use custom certificate to authenticate the User-ID agent's Server Certificate.
   1. Create a Root Certificate in the firewall by navigating to Device > Certificate Management > Certificates > Generate and fill out the following parameters:
      1. Certificate Name: MY_ROOT_CA
      2. Common Name: Use either the NGFW management IP Address or its FQDN 
      3. Certificate Authority: check 
   2. Then, click Generate.

1. 3. Configure the newly generated Certificate to be a Trusted Root CA, by navigating to Device > Certificate Management > Certificates > MY_ROOT_CA > Trusted Root CA: check > OK.

1. 4. Generate User-ID agent's Server Certificate, go to Device > Certificate Management > Certificates > Generate and fill out the following parameters:
      1. Certificate Name: USER-ID-AGENT-CERT
      2. Common Name: Use either the User-ID agent's IP Address or its FQDN 
      3. Signed By: MY_ROOT_CA

1. 5. Export the User-ID agent's public and private certificates from the NGFW, Device > Certificate Management > Certificates > USER-ID-AGENT-CERT:check > Export Certificate, use the following parameters as seen below:

1. 6. Import the User-ID agent's private/public certificate to the host/server.
   7. Configure the User-ID agent's server certificate, by going to Server Certificate > Add.

1. 8. Save the changes by clicking on Save then Commit