How to troubleshoot connection failure between Firewall MP and Application Cloud Engine (ACE)

How to troubleshoot connection failure between Firewall MP and Application Cloud Engine (ACE)

1596
Created On 08/03/25 05:48 AM - Last Modified 08/04/25 20:45 PM


Objective




Environment


  • Palo Alto Firewalls
  • PAN-OS 12.1 and above.
  • App-ID Cloud Engine (ACE)
  • Knowledge Centered Service Cloud


Procedure


  1. Check that a device certificate is valid and present on the FW.

    show device-certificate status
  2. Check that the SaaS Security Inline license is present and Valid.

    request license info
  3. Check that the Data Services service route is properly configured. (Default is management)

  4. Check if the upstream firewall is allowing application paloalto-ace, paloalto-ace-kcs, paloalto-dlp-service and OCSP (for certificate validation).

    1. The following FQDNs need to be allowed in a security policy for certificate verification:

      • ocsp.paloaltonetworks.com
      • crl.paloaltonetworks.com
      • ocsp.digicert.com
      • crl.digicert.com
      • crl3.digicert.com
      • crl4.digicert.com
      • ocsp.godaddy.com
      • crl.godaddy.com
  1. Make sure that the App-ID cloud engine is enabled on the Firewall (it is enabled by default on the FW).

  2. Troubleshoot the connection between Firewall Management Plane (MP) and App-ID Cloud Engine (ACE):

    1. Check the cloud connection status to the Firewall MP.

      show cloud-appid connection-to-cloud

Note: In addition to the connection status, this output will help guide your troubleshooting by indicating whether the problem is the device certificate or the missing license as explained here.

    1. Check the network connection between the Firewall Data Services service route, source IP, and the ACE server, destination FQDN:

      traceroute host kcs.ace.tpcloud.paloaltonetworks.com

Note: This command is valid in case Management is used as Data Services service route and the kcs.ace.tpcloud.paloaltonetworks.com is the FQDN of the ACE server found in the output of 6.a. Otherwise add source to the command followed by the IP address of the dataplane interface used as service route.

    1. Check if connection is established on port 443 between the Firewall and the ACE server:

      show netstat numeric-hosts yes numeric-ports yes | match x.y.z.q

Here replace x.y.z.q by the  IP address of the ACE server resolved by the DNS server in 6.b

    1. Check Firewall system logs related to this MP connection:

      show log system subtype equal app-cloud-engine direction equal backward
    1. As last resort and if needing to restart the connection between FW MP and ACE server use:

      debug cloud-appid reset connection-to-cloud


Additional Information


Prepare to Deploy APP-ID Cloud Engine 

Troubleshoot APP-ID Cloud Engine  

The Firewall maintains two connections to the cloud: One connection from Firewall MP to ACE server and another connection from Firewall DP to Content Cloud server. This document focuses on the MP connection.
Below is an example of the output of a good connection between FW and ACE.

> show cloud-appid connection-to-cloud

ACE Cloud server: kcs.ace.tpcloud.paloaltonetworks.com:443
Cloud connection: connected

Summary of ACE gRPC client:
number of connection reset:       0
number of connection failed:      110
number of connection established: 1685
number of connection attempts:    1795
number of connection released:    1684
number of connection selected:    3248
number of selections failed:      8
number of bytes sent:             183674
number of bytes received:         207531676
Last gRPC connection Attempt:     2025-08-01 14:01:39 -0700 PDT
Last successful gRPC connection:  2025-08-01 14:01:39 -0700 PDT

Summary of gRPC connections [configured source IP: ]
Device cert status: Installed
Validity:
                Notbefore: 2025-07-17 07:41:13 +0000 UTC
                Notafter: 2025-10-15 07:41:12 +0000 UTC

max gRPC connections: 1, ongoing: 0, max alive time: unlimited, max bytes sent: unlimited
[0]gRPC conn[10.6.158.66:42730 -> 34.102.229.209:443], state true, selected 0, backup false,0, device cert
send: wire 0, app 0, num 0;  receive: wire 0, app 0, num 0
Current Time:  2025-08-01 14:02:39.424805538 -0700 PDT m=+1383066.393353224


Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA1Ki000000kA7TKAU&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail