How to mitigate High DP CPU issue due to an increase in URL cache processing
10817
Created On 05/31/23 22:21 PM - Last Modified 04/22/24 05:52 AM
Objective
To mitigate High DP CPU issue due to an increase in URL cache processing. This root cause of high DP CPU would have been determined by noticing that the increase in DP CPU coincided with an increase in total-us value of urlcache_lookup function which represents the total processing time in microseconds μs for this function in the last sampling cycle seen in the output of "debug dataplane pow performance".
Environment
- Palo Alto Firewall
- DP CPU
- URL categorization
Procedure
- Check If high DP CPU and Packet Descriptor (on chip) are seen in the output of:
show running resource-monitor <minute/hour/second> last <time>
- Check If URL cache lookup processing time is going high:
debug dataplane pow performance | match "func\|urlcache_lookup"
Look for any spikes in the output for the total-us or max-us value. - Check if the global counter "url_request_pkt_drop" is also seeing an increase in the output of:
show counter global | match url_request_pkt_drop
- Query the URL filtering logs for ((category eq not-resolved) or (category eq unknown)) under MONITOR > Logs> URL Filtering this will help find if a certain domain is causing the problem.
- In the case where a domain with URL category not-resolved or unknown is found, then run the CLI commands below against that domain:
test url-info-host <domain name>
to query the MP URL database.show running url-info <domain name>
to query the DP URL database. - If this is an internal trusted domain then consider the configuration of an application override policy to all trusted traffic sent to this domain's hosts IPs or FQDNs (Note that app-override disables all security inspection).
- Clear MP and DP cache from that particular domain:
- To delete the MP cache for that domain run CLI command:
delete url-database url <url>
- To delete the DP cache for that domain run CLI command:
clear url-cache url <url>
- To delete the MP cache for that domain run CLI command:
- In case you need to open a support ticket, paste the output of below commands run 3-5 times (before clearing the cache) in the case comment:
show running url-cache statistics show running url-info <domain name> test url-info-host <domain name> debug dataplane pow performance | match "func\|urlcache_lookup"
- In the case where a domain with URL category not-resolved or unknown is found, then run the CLI commands below against that domain:
and generate a techsupport file and attach it to the ticket.
- If the FW is running PAN-OS version lower than 11.0.0, 10.2.4, 10.1.9, 10.1.8-h3, then upgrade the FW to those PAN-OS versions or to higher versions which have the fix to below known issue:
- PAN-174953 Fixed an issue where the firewall didn't update URL categories from the management plane to the dataplane cache.