How to troubleshoot DP performance issue due to session table exhaustion
12187
Created On 06/21/23 17:08 PM - Last Modified 08/23/23 22:45 PM
Objective
To root cause and mitigate DP performance issues due to the number of concurrent sessions reaching the FW's capacity limit. This root cause of DP performance issue would have been determined by noticing that the increase in dropped connections across the FW's dataplane coincided with session table utilization increasing and reaching the FW’s supported limit.
Environment
- Palo Alto Firewall
- DP performance
- Session table exhaustion
Procedure
- Determine the source IP, destination IP and application of the traffic flows which are taking up the highest number of concurrent sessions:
- To find the traffic with terminated sessions:
- Search the traffic logs MONITOR > Logs > Traffic using the timestamp of high DP CPU's detection with the filters: (receive_time leq <time value>) and (receive_time geq <time value>).
- Check the ACC tab and filter using the timestamp of detection of high DP CPU Time > Custom > Time Range; Search for the traffic that has high number of sessions select the radio button sessions in all the ACC widgets.
- To find the traffic with active sessions:
- Check the MONITOR > Session Browser.
- Check the output of CLI command:
show session all
Refer to How to Monitor Live Sessions in the CLI
- To find the traffic with terminated sessions:
- If the surge in traffic sessions is due to a scheduled activity like scheduled updates, data polling or data synching then consider to set that schedule outside normal production hours meaning outside peak traffic hours.
- If the overflow of traffic needs to be stopped then try to stop it at its source otherwise it is best to use Zone protection and DOS protection configuration on the FW.
- If the reason of the session table utilization reaching its highest capacity is due to the fact that session timeout setting for this type of traffic is set too high or too low by default in that case refer to Tips & Tricks Session Timeout.
- In the case where none of the above steps can be used to reduce the session table utilization below the capacity limit of the FW, the sessions received by the FW are eligible and the session timeout settings are valid, then consider upgrading your FW to a higher capacity platform.
Additional Information
As best practice and in order to protect the firewall's resources it is always recommended to enable packet buffer protection globally under Device > Setup > Session Settings and per zone under Network > Zones.