Palo Alto Networks Knowledgebase: How to Open a Support Case on Routing Issues (OSPF and BGP)

How to Open a Support Case on Routing Issues (OSPF and BGP)

19215
Created On 09/25/18 19:48 PM - Last Updated 08/05/19 19:48 PM
Mobile Network Infrastructure
Resolution

How to open a case on routing issues

This document shows a few common issues with the OSPF routing protocol and shows how to collect preliminary information required by the Technical Assistance Center (TAC) to start working on routing issues:

 

OSPF routing and solutions

Let's look at a few case studies that document common issues with OSPF routing and solutions.

OSPF Case Studies
StateWhat to Check
Stuck between Init and Two-Way state.Look for Duplicate Router ID.
Stuck in Ex/Start.Look for MTU mismatch. Devices between the PA firewall and the Neighboring device can have mismatched MTU as well. (See Wireshark screen below).
Stuck in Exchange State: Master hasn’t received an ACK from Slave. Sequence Numbers don’t match (The initial DD Sequence number value, which is indicated by the Init bit being set, should be unique. The DD seq. number then increments until the complete database description has been sent.)Look for any unicast reachability issues. Use pings to verify connectivity, and check the ARP entry.
Stuck in Loading State: No LS Update Received or no LS Ack received.Look in the packet capture to confirm this.
Neighbor state flaps from Full to Init.Look for Inconsistent BFD configuration. PAN-OS supports BFD starting with PAN-OS 7.1.
Authentication mismatch.Look in routed logs to verify any authentication mismatch issues.
Hello Packet parameters mismatch between neighbors.Look in routed logs for Hello Interval value mismatch (See routed log screen below).
Routing table is full, no resources are available for route installationUse the command "> show routing resource or summary" to look at resource useage (See resource usage screen below).


open case routing 1.pngWireshark screen showing packet info
open case routing 2.pngRouted.log showing hello interval

open case routing 3.pngResource Usage output from the >show routing summary command

Other OSPF issues

 For any other OSPF issues, follow the steps below to collect the required information.

  1. Get a snapshot of the memory and CPU utilization of each process in the system.
    > show system resources
  1. Enable debug on “routed” daemon to get additional information in the logs. Please note that enabling debug on routed daemon might increase the CPU utilization.
    > debug routing global on debug
  1. Take a packet capture of the OSPF packets on the data plane. Set up the packet filters to capture traffic over IP protocol 89.
    please see: Getting Started: Packet Capture
  1. Take a packet capture of the OSPF packets on the management plane.
    > debug routing pcap ospf delete
    > debug routing pcap ospf on
  1. Check the state of the OSPF sessions.
    > show session all filter protocol 89
    > show session id <value from previous command>
  1. Check the ARP table to verify the MAC addresses of the OSPF neighbors.
    > show arp all
  1. Verify the egress interface used to reach the OSPF neighbor.
    > test routing fib-lookup virtual-router <vr-name> ip <neighbor IP>
  1. Print the time to help correlating the output of these commands with the data in the Tech Support file.
    > show clock
    > show counter global filter delta yes packet-filter yes (run this multiple times during the time of the issue)
  1. Check the OSPF summary information.
    > show routing protocol ospf summary
  2. Wait until the issue happens, so that the relevant data can be captured then turn off debugs and pcaps.
    > debug dataplane packet-diag set capture off
    > debug routing pcap ospf off
    > debug routing global on info
  3. Export OSPF pcap from the management plane and attach it to the case.
     > scp export debug-pcap from "file name" to username@host:path
    Or
    > tftp export debug-pcap from "file name" to "tftp host"
  4. Download OSPF pcap done on the data plane from Monitor > Packet Capture tab on the Web UI and attach it to the case.
  5. Provide a network topology.
  6. Provide OSPF configuration from the neighbor device(s).
  7. Generate a Tech Support file and attach it to the case.
    Please see How to Generate and Upload a Tech Support File Using the WebGUI and CLI
  8. Please specify the exact time of the issue.

 

BGP

For BGP routing issues, please follow the below steps to collect the required data:

  1. Get a snapshot of the memory and CPU utilization of each process in the system.
    > show system resources
  1. Enable debug on “routed” daemon to get additional information in the logs. Please note that enabling debug on routed might increase the CPU utilization.
    > debug routing global on debug
  1. Take a packet capture of the BGP packets on the data plane. Set up the packet filters to capture traffic over TCP port 179.
    Please see: Getting Started Packet Capture
  1. Take a packet capture of the BGP packets on the management plane.
    > debug routing pcap bgp delete
    > debug routing pcap bgp on
  1. Check the state of the BGP sessions.
    > show session all filter destination-port 179
    > show session id <value from previous command>
  1. Check the ARP table to verify the MAC addresses of the OSPF neighbors.
    > show arp all
  1. Verify the egress interface used to reach the BGP neighbor.
    > test routing fib-lookup virtual-router <vr-name> ip <neighbor IP>
  1. Print the time to help correlating the output of these commands with the data in the Tech Support file.
    > show clock
    > show counter global filter delta yes packet-filter yes
    (run this multiple times during the time of the issue)
  1. Check the BGP summary information.
    > show routing protocol bgp summary
  2. Wait until the issue happens, so that the relevant data can be captured then turn off debugs and pcaps.
    > debug dataplane packet-diag set capture off
    > debug routing pcap bgp off
    > debug routing global on info
  3. Export BGP pcap from the management plane and attach it to the case.
     > scp export debug-pcap from "file name" to username@host:path
    Or
    > tftp export debug-pcap from "file name" to "tftp host"
  4. Download BGP pcap done on the data plane from Monitor > Packet Capture tab on the Web UI and attach it to the case.
  1. Provide a network topology
  2. Provide BGP configuration from the neighbor device(s).
  3. Generate a Tech Support file and attach it to the case.
    Please see: How to Generate and Upload a Tech Support File Using the WebGUI and CLI
  4. Please specify the exact time of the issue.

Why do we need this information?

While troubleshooting routing issues, it is important to make sure that the Palo Alto Networks firewall is able to receive and transmit packets involved in establishing routing adjacencies, therefore, having a packet capture on the data plane as well as on the management plane is very helpful for troubleshooting.

 

The packet capture on the management plane will confirm that routed daemon receives routing protocols packets successfully from the data plane. Enabling debug on the routed process provides more details required for troubleshooting.

 

author: Alaauddin Shieha 



Attachments
Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000CldcCAC&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail

Attachments
Choose Language