Error Message: x509: certificate has expired or is not yet valid is seen on Prisma Cloud Compute

27070

Created On 01/18/22 18:18 PM - Last Modified 04/22/22 18:10 PM

Console

Threat Intelligence

Self-Hosted 20.04

Self-Hosted 20.09

Prisma Cloud Compute Edition Self-Hosted

Symptom

All defenders show disconnected from the console.
Defender logs show the error "x509: certificate has expired or is not yet valid: current time 2021-12-29T01:51:15Z is after 2021-11-13T17:59:00Z"

ERRO 2021-12-29T01:51:15.343 defender.go:1455 No console connectivity wss://10.10.10.10:8084
ERRO 2021-12-29T01:51:15.482 api.go:230 Receive connection error Get " https://10.10.10.10:8084/api/v1/dbip-country-lite.mmdb": Forbidden; retrying without proxy
ERRO 2021-12-29T01:51:15.489 defender.go:1578 failed to init geoip db failed to download geoip db Get " https://10.10.10.10:8084/api/v1/dbip-country-lite.mmdb": x509: certificate has expired or is not yet valid: current time 2021-12-29T01:51:15Z is after 2021-11-13T17:59:00Z

Environment

Prisma Cloud Compute Self-Hosted
Any version.

Cause

In order to establish secure communication between the Defender and Console TLS certificates are used.
There is a two-way SSL handshake that occurs between the Defenders and Console.
This two-way handshake requires that both components (Defenders and Console) have certificates issued by the same Certificate Authority.
The Console generates the self signed certificate bundle and it also serves as the repository for all the certificates used in the deployment. When the CA certificate (root certificate) expires, a new certificate bundle needs to be generated.
Once this new bundle is created the appropriate certificates are made active upon the Console. After this occurs the Defenders would no longer be able to communicate with the Console because the certificates they have are no longer valid.
The two-way SSL handshake between Defenders and Console can no longer be established until this certificate issue is resolved. This is why the Defenders show up as disconnected in the Console.

Resolution

Note: The newer versions of the Console (21.08 and above) renew the certificates automatically. This document is applicable in case the certificates are not renewed automatically

Option1:
If running versions below 21.08 or if upgrade is not currently possible or if the certificates are not renewed automatically, one may use the API to rotate and renew the certificate manually.

Backup the certs directory from your console (default location is /var/lib/twistlock/certificates)
The API call to trigger a certificate rotate (to refresh the expiry) is:

curl -k -u "USER:PASSWORD" -X PUT https://console:8083/api/v1/certs/rotate

   Reference: API Documentation of Twistlock

After running it successfully, you should see the timestamps of the certificate files reflect the change.
Re-deploy your defenders.

**New certificates would mean defenders would also need to be re-deployed so they may register and use the refreshed certs to re-connect to the console.

Option2:

If already running 21.08, the Prisma Cloud Compute Console will display a banner indicating that the certificates are nearing expiration.

The banner pops up automatically 30 days prior to expiry.
In this same banner, a clickable link called "Rotate Certificates" will be visible.

Click on this link to automatically renew the certificates.
Re-deploy your defenders.

Reference: Custom Certificate For Console

Additional Information

To view details of current certificates under /var/lib/twistlock/certificates, one may use openssl or any similar tool as shown below.

# openssl s_client -connect console:8083 > certstatus.txt