Slow commits and high Management Plane CPU when Load Balancer health checks use TCP port 22 on VM-Series
49244
Created On 05/04/21 00:40 AM - Last Modified 06/26/26 20:33 PM
Symptom
- Higher than expected commit times
- Longer time for CLI and GUI access
- High Management Plane (MP) CPU
Environment
- VM-Series
- Cloud deployments using a Load Balancer (e.g., GCP, Azure, AWS...)
- Management profile on the Dataplane (DP) interface (or loopback) enabled for SSH, HTTP, or HTTPS
- VM-Series responding to Health Checks
Cause
Some load balancers, such as GCP, use multiple probers for health checks. SSH launches internal shells for each connection request, which takes a heavy toll on Management Plane resources. Modifying the interval and timeout could reduce the number of probes to the Management Plane to 5 to 15 connections, but the resource toll remains. HTTP does not have the same overhead for the Management Plane as SSH connections do.
Resolution
- Avoid using TCP port 22 for health checks.
- Use HTTP and direct probes to /unauth/php/health.php (Available in PAN-OS 11.0+, 10.2.4+, and 10.1.9+).
- Direct the probes to
/php/login.phpfor other PAN-OS versions. (Note: Aggressive probes to login.php may result in slow GUI response.) - If using TCP port 22 is inevitable, use a higher time interval (such as 30-60s) for the probe interval to minimize Management Plane impact.
Additional Information
- GCP load balancer health check concepts (Multiple Probers): https://cloud.google.com/load-balancing/docs/health-check-concepts#multiple-probers
- GCP load balancer decision tree: https://cloud.google.com/load-balancing/docs/choosing-load-balancer
- Securing applications in Azure: https://www.paloaltonetworks.com/apps/pan/public/downloadResource?pagePath=/content/pan/en_US/resources/guides/azure-transit-vnet-deployment-guide