The firewall starts to drop a UDP traffic which was allowed earlier, even there has no changes made in the firewall config or anywhere in the network

69363

Created On 01/12/21 16:26 PM - Last Modified 02/22/24 09:32 AM

PAN-OS

Strata

Symptom

There may be a situation like PAN firewall starts blocking a UDP traffic (i.e. ESP, DHCP, DNS, NTP) which had been allowed earlier.
This is possible even there has been no changes made in the firewall config or anywhere in the environment.

Environment

All hardware and VM platforms
All PanOS releases

Cause

PAN firewall creates and uses session records while processing the traffic passing through. The article below can be checked regarding how firewall handles the traffic. https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClVHCA0

The sessions can be in various states. The article below can be checked regarding the description of the session states.
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000Clk9CAC

If a session is in "Discard" state, any packet received by the firewall and hit that session is always dropped.

A session may stuck in "Discard" state in certain scenarios. The scenarios may be but not limited to the followings:

Example Scenario 1: An IPSEC tunnel is failing after a IPSec config change or system restart
- In this scenario the other IPSec peer may continue to send ESP packets during the system restart.
- Per the policies/config in place traffic is dropped by the firewall and a session which was initially created goes into "Discard" state.
- All following packets hit the same session and the session remains and stay stuck in "Discard" state.
- A new phase2 never gets established because all messages hit the same session and are dropped.

Example Scenario 2: A relayed DHCP traffic starts to fail all of a sudden
- In this scenario hardware offloading that we have in most of our hardware models plays role.
- An existing session related to the DHCP traffic may time out on DP due to our offloading logic. Please check the article below regarding how and when network processor updates the session timers on DP.
https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000Cm8cCAC
- If the DHCP traffic is allowed from Zone A to Zone B and if the session times out before the response coming from Zone B to Zone A, this response message will be dropped and there will be a session seen in "Discard" state.
- The following packets will hit this this session and will be dropped.

Resolution

To resolve the drops on the firewall a few actions can be taken:

The sessions in "Discard" state can be listed and removed by the following commands. It is important to look for the sessions for each direction.

> show session all filter source [IP address 1] destination [IP address 2] state discard
> show session all filter source [IP address 2] destination [IP address 1] state discard
> show session all filter state discard
> clear session id [Session Id]

In scenarios where session timeout is also effective, the session timeout value can be increased for the necessary app-ids. This can be done by changing the UDP timeout value on GUI Objects > Applications > [Concerned App-id] > UDP Timeout (Second) on the WebUI.

If changing the session timeout would not help, then another option could be defining an allow rule for the reverse direction. (Referring to the example scenario 2, an allow rule from Zone B to Zone A) So the return traffic will trigger a brand new session and will not be dropped.

Additional Information