Traffic drops for 120 seconds during BGP link failover with BFD and Graceful Restart enabled
Symptom
-
Experiencing a ~2-minute (120 seconds) traffic drop or blackhole when a physical link or interface between the firewall and a BGP peer (e.g., downstream router) goes down.
-
Outbound traffic from the downstream router fails over instantly to a secondary path.
-
Return traffic from the firewall is dropped for the duration of the Graceful Restart timer.
-
BFD (Bidirectional Forwarding Detection) is configured and tears down the BGP session immediately, but sub-second routing failover is not achieved.
Environment
-
Palo Alto Networks NGFW
-
PAN-OS
-
BGP (Border Gateway Protocol)
-
BFD (Bidirectional Forwarding Detection) enabled on the peering
-
BGP Graceful Restart (GR) enabled on both peers
Cause
This issue is caused by a design conflict between the fast-failover mechanism of BFD and the stability-focused mechanism of BGP Graceful Restart.
When the link fails, BFD detects the failure in milliseconds and signals BGP to tear down the session. Because Graceful Restart is negotiated between the peers, the firewall interprets the BFD session drop as a temporary control-plane restart of the peer router.
The firewall immediately enters Graceful Restart Helper Mode. It deliberately retains the stale BGP routes pointing to the down interface for the configured Graceful Restart timer (default 120 seconds) to prevent routing churn. While the downstream router instantly reroutes outbound traffic, the firewall forwards return traffic back to the dead interface until the timer expires, causing an asymmetric traffic blackhole.
Resolution
To prevent the 120-second delay during HA failover event, BFD and BGP Graceful Restart should not be used together for the same BGP peering.
Implement one of the following configuration changes:
Option 1: Disable BFD on BGP globally, and keep BGP Graceful Restart (GR) enabled.
-
Log in to the Palo Alto Networks firewall web interface.
-
Navigate to Network > Virtual Routers.
-
Click the name of the Virtual Router handling the BGP peering.
-
On the left pane, click BGP
-
Locate the BFD section at the top of the BGP configuration window
-
Select None to disable BFD globally.
Option 2: Disable Graceful Restart globally for the BGP Peering, and keep BFD enabled. Once BGP GR is disabled, BFD will have sole authority to tear down the session. Routes will be flushed instantly upon a link-down event.
-
Log in to the Palo Alto Networks firewall web interface.
-
Navigate to Network > Virtual Routers.
-
Click on the name of the Virtual Router handling the BGP peering.
-
On the left pane, click BGP, then select Peer Group.
-
Click on the specific Peer Group (or the individual Peer) connecting to the router.
-
Navigate to the Advanced tab within the Peer or Peer Group configuration window.
-
Locate the Graceful Restart settings.
-
Uncheck the Enable box for Graceful Restart to disable it entirely for this specific peer/group. (Note: If it is currently inheriting the setting, you may need to override the inheritance first).
-
Click OK, and then Commit the changes.
Additional Information
Increasing BFD timers (Desired Minimum Tx Interval and Detection Time Multiplier) will not resolve this specific issue, as the firewall will still enter Graceful Restart Helper Mode once the delayed BFD session drop occurs.