如何解决防火墙与 ACE 应用云引擎/内容云之间的 gRPC 连接失败问题

• Palo Alto 防火墙
• PAN-OS 10.1 及以上版本。
• App-ID 云引擎 (ACE)
• 内容云（文件管理器）

1. Check that a device certificate is valid and present on the FW.

   show device-certificate status

2. Check that the SaaS Security Inline license is present and Valid.

   request license info

3. 检查数据服务服务路由是否配置正确。（默认为管理）
4. 检查上游防火墙是否允许应用程序paloalto-ace、paloalto-ace-kcs 和OCSP （用于证书验证）。
5. 确保防火墙上启用了 App-ID 云引擎（FW 上默认启用）。
6. Troubleshoot the connection between Firewall Management Plane (MP) and App-ID Cloud Engine (ACE):
   1. Check the cloud connection status to the Firewall MP.

      show cloud-appid connection-to-cloud   

      1. 注意：除了连接状态之外，此输出还将通过指示问题是设备证书还是缺少许可证来帮助指导您的故障排除，如此处所述。
   2. Check the network connection between the Firewall Data Services service route, source IP, and the ACE server, destination FQDN:

      traceroute host kcs.ace.tpcloud.paloaltonetworks.com

      1. 注意：如果将管理用作数据服务服务路由，并且 kcs.ace.tpcloud.paloaltonetworks.com 是在 5.a 的输出中找到的 ACE 服务器的FQDN ，则此命令有效。否则，将源添加到命令中，后跟用作服务路由的数据平面接口的IP 地址。
   3. Check if connection is established on port 443 between the Firewall and the ACE server:

       show netstat numeric-hosts yes numeric-ports yes | match 34.120.110.215

      1. 其中 34.120.110.215 是B中DNS服务器解析的 ACE 服务器的IP 地址
   4. Check Firewall system logs related to this MP connection:

      show log system subtype equal app-cloud-engine direction equal backward

   5. As last resort and if needing to restart the connection between FW MP and ACE server use:

      debug cloud-appid reset connection-to-cloud

7. Troubleshoot the connection between Firewall Data Plane (DP) and Content Cloud (filemanager):
   1. Check the cloud connection status to the Firewall DP.

      show ctd-agent status security-client

      1. 注意：在“ Security Client Ace ”部分下，云连接应显示已连接，且池状态应显示就绪 (2) 。
   2. Check the network connection between the Firewall Data Services service route, source IP, and the Content Cloud (filemanager) server, destination FQDN:

      traceroute host ace.hawkeye.services-edge.paloaltonetworks.com

      1. 注意：如果将管理用作数据服务服务路由，并且 ace.hawkeye.services-edge.paloaltonetworks.com 是 6.a 输出中找到的内容云服务器的FQDN ，则此命令有效。否则，将源添加到命令，后跟用作服务路由的数据平面接口的IP 地址和根据您的区域添加正确的FQDN 。
   3. Check if connection is established on port 443 between the Firewall and the Content Cloud (filemanager) server:

       show netstat numeric-hosts yes numeric-ports yes | match 34.111.222.75

      1. 其中 34.11.222.75 是B中DNS服务器解析的内容云服务器的IP 地址
   4. Check Firewall system logs related to this DP connection:

      show log system subtype equal ctd-agent-connection direction equal backward

   5. As last resort and if needing to restart the connection between FW DP and Content Cloud server use the following CLI with great caution knowing that it is very disruptive as it also affects other Firewall's inline cloud analysis services (IOT, enterprise DLP, advanced URL filtering).

      debug software restart process ctd-agent

      1. 注意：重新启动 ctd-agent 将重置防火墙 DP 和内容云服务器之间的连接。

> show ctd-agent status security-client

Security Client Ace(1)
        Current cloud server:   ace.hawkeye.services-edge.paloaltonetworks.com:443
        Cloud connection:       connected
        Config:
                Number of gRPC connections: 2, Number of workers: 5
                Debug level: 2, Insecure connection: false, Cert valid: true, Key valid: true, CA count: 383
                Maximum number of workers: 10
                Maximum number of sessions a worker should process before reconnect: 1024
                Maximum number of messages per worker: 0
                Skip cert verify: false
        Grpc Connection Status:
                State Ready (3), last err <nil>
                Pool state: Ready (2)
                     last update: 2023-04-10 11:29:18.888023715 -0700 PDT m=+330284.693222939
                     last connection retry: 2023-04-10 09:27:50.049422541 -0700 PDT m=+322995.854622294
                     last pool close: 2023-04-10 09:27:32.141236266 -0700 PDT m=+322977.946435573