Existing sessions hitting incorrect NAT rule after PBF failover or route change
17380
Created On 04/25/19 09:09 AM - Last Modified 05/14/19 00:06 AM
Symptom
Static route path monitoring or Policy Based Forwarding (PBF) rules on the firewall are commonly used to configure route failover. This is commonly configured for dual/multiple Internet Service Providers (ISP) environments.
For more information on dual/multiple ISP configuration, refer to the article Dual ISP redundancy using Static Routes Path Monitoring Feature, for Traffic Failover.
Symptoms:
- Long lived sessions (such as SIP, IPSec) start losing traffic due to change in routing or enable/disable of PBF rule
- Triggered by path monitoring failure
Environment
- All PAN-OS
- Firewall terminating multiple ISP connections and has individual NAT rule configuration for each of the ISP interfaces.
- Multiple ISP scenarios for both Static route path monitoring and PBF rules
Cause
After the route failover, the firewall does not update its current sessions with the new NAT rule.
For example, assume that the static route path monitoring is used for route failover between two ISP routes.
Primary route is through ethernet1/4 and secondary route is through ethernet1/5.
There are two separate NAT rules configured for the Source NAT (SNAT) of the traffic through both the ISP interfaces (ethernet1/4 and ethernet1/5).
Session info before path monitoring failover:
> show session info 108876 Session 108876 c2s flow: source: 192.168.1.117 [L3 - Trusted ] dst: <Server's Public IP> proto: 17 sport: 5060 dport: 5060 state: ACTIVE type: FLOW src user: unknown dst user: unknown s2c flow: source: <Server's Public IP>[L3 - Untrusted] dst: 10.75.75.78 <<<< Primary ISP Interface IP proto: 17 sport: 5060 dport: 39176 state: ACTIVE type: FLOW src user: unknown dst user: unknown ... nat-rule : NAT-PrimaryInternet(vsys1) <<<< NAT rule for Primary ISP ingress interface : ethernet1/3 <<<< LAN interface egress interface : ethernet1/4 <<<< Primary WAN interface
Consider there is a path monitoring failure on ethernet1/4. The default route shifts to the Secondary ISP circuit, which routes all traffic through ethernet1/5.
Since the traffic is now routed through ethernet1/5, the NAT rule that should be hit is NAT-SecondaryInternet. However, the session info is not updated to use the new NAT policy and egress interface. Hence, the session info after the route change still has the same NAT policy and egress interface.
Session info after path monitoring failover:
> show session info 108876 Session 108876 ... nat-rule : NAT-PrimaryInternet(vsys1) <<<< NAT rule for Primary ISP ingress interface : ethernet1/3 <<<< LAN interface egress interface : ethernet1/4 <<<< Primary WAN interface
After the route failover, the packets continue to hit the NAT policy NAT-PrimaryInternet which is translating to the Primary ISP interface IP. This causes the packets to be translated with the incorrect source IP address when forwarded to the secondary circuit through ethernet1/5 (Secondary ISP Interface).
Resolution
- This is expected behavior on the PA firewall.
- The sessions will have to be manually cleared to fix the traffic flow.
It can be cleared using the below command.
> clear session id <session ID>