Diagnosing OSPF Database Route Installation Issues
Symptom
- Firewalls PA-1410 and PA-820 peer with each other in the backbone area (Area 0.0.0.0) as follows:
admin@PA-1410> show routing protocol ospf neighbor
Options: 0x80:reserved, O:Opaq-LSA capability, DC:demand circuits, EA:Ext-Attr LSA capability,
N/P:NSSA option, MC:multicase, E:AS external LSA capability, T:TOS capability
==========
virtual router: default
neighbor address: 10.10.10.2
local address binding: 0.0.0.0
type: dynamic
status: full
neighbor router ID: 2.2.2.2
area id: 0.0.0.0
<output intentionally removed>admin@PA-820> show routing protocol ospf neighbor
Options: 0x80:reserved, O:Opaq-LSA capability, DC:demand circuits, EA:Ext-Attr LSA capability,
N/P:NSSA option, MC:multicase, E:AS external LSA capability, T:TOS capability
==========
virtual router: default
neighbor address: 10.10.10.1
local address binding: 0.0.0.0
type: dynamic
status: full
neighbor router ID: 1.1.1.1
area id: 0.0.0.0
<output intentionally removed>- Firewall PA-850 is advertising the route 10.20.0.0/24 to the backbone area (as a type-3 summary prefix.) This route appears in the link state database (LSDB) of PA-1410; but the route is missing from its routing table:
admin@PA-1410> show routing protocol ospf dumplsdb
VIRTUAL ROUTER: default (id 1)
==========
VR Area ID Orig RTR ID LS ID LSA Type Seq Number CheckSum Age Size
<output intentionally omitted>
1 0.0.0.0 2.2.2.2 10.20.0.0/24 type-3 (Summary) 0x80000006 0x00001612 724 28
Options: [External]
Mask 255.255.255.0, metric: 10
<output intentionally omitted>
admin@PA-1410> show routing route | match 10.20.0.0/24
admin@PA-1410>
Environment
- All versions of PAN-OS
- OSPF Routing
- Hardware NGFW
- VM-Series NGFW
Cause
Comparing the LSAs between PA-1410 and PA-850, it can be observed that these two OSPF neighbors are generating different link information in the LSA. PA-1410 is describing the link with PA-850 as a stub network; while PA-850 describes the same link peering to PA-1410 as a transit link.
The output below is taken from PA-1410 but it is the same output seen on PA-850 -- which is expected, since all OSPF peers maintain the same link-state database (LSDB) information.
admin@PA-1410> show routing protocol ospf dumplsdb
VIRTUAL ROUTER: default (id 1)
==========
VR Area ID Orig RTR ID LS ID LSA Type Seq Number CheckSum Age Size
1 0.0.0.0 1.1.1.1 1.1.1.1 type-1 (Router) 0x8000003B 0x00000B9F 469 48
Options: [External]
Router LSA Options: [ASBR]
P2P Link: Neighbor (ID: 2.2.2.2) on Interface Index 10.10.10.1, tos 0, metric: 10
Stub Network: 10.10.10.0 Netmask 255.255.255.252, tos 0, metric: 10
1 0.0.0.0 2.2.2.2 2.2.2.2 type-1 (Router) 0x8000003C 0x0000F4C6 455 36
Options: [External]
Router LSA Options: [ABR, ASBR]
Transit Network: DR (IP: 10.10.10.2) on Interface 10.10.10.2, tos 0, metric: 10
<output intentionally removed>This leads to a mismatch in the LSDB -- resulting in failure to install any routes exchanged between the peers. Looking further at these particular links on each respective peers, it is evident that they are configured as different link-types.
PA-1410:
admin@PA-1410> show routing protocol ospf interface
==========
virtual router: default
interface name: tunnel.1
interface address: 10.10.10.1
interface type: p2p
passive: no
area id: 0.0.0.0
<output intentionally removed>PA-850:
admin@Lab32-13-PA-820-ZTP> show routing protocol ospf interface
==========
virtual router: default
interface name: tunnel.1
interface address: 10.10.10.2
interface type: broadcast
passive: no
area id: 0.0.0.0
<output intentionally removed>Because of the mismatch in interface type between these peers (broadcast vs point-to-point,) the routes will eventually fail to install in the RIB/routing table despite being in the LSDB. The reason is because these link types use different LSA types to solve the shortest path first (SPF) tree. For example, with a point-to-point link type there is no DR/BDR election as is the case with broadcast link-types (point-to-point designates one-to-one communication.)
Resolution
Both peers should be configured with the same link-type. This is one of the reasons why OSPF is considered a link-state protocol. In the scenario outlined on this document, the ideal solution to the problem is to configure each tunnel interface as a point-to-point (p2p) since this is the recommended/ideal interface type for tunnel links (such as GRE and IPsec.)
After applying a matching link-type on PA-850, PA-1410 is able to install the prefix in its routing table with a next hop pointing to PA-850:
admin@PA-1410> show routing route | match 10.20.0.0/24
10.20.0.0/24 10.10.10.2 20 A Oo 13 tunnel.1
Additional Information
The following attributes must match between OSPF adjacencies -- with few exceptions in very specific cases:
- Interface Area-ID
- Interface MTU *
- Interface network subnet
- Hello and dead interval
- Network/interface type *
- Authentication
- Stub flags
- Other optional capabilities
Unlike other dynamic protocols such as BGP (which operates as destination-based routing,) OSPF routes packets between the nodes in the link-state graph by solving the shortest path to the nodes. Once an OSPF speaker knows the shortest path to all the nodes in the OSPF graph/tree, it means that it implicitly knows the shortest path to the attributes (prefixes) on the links of the respective nodes -- since it has already solved the shortest path to all nodes in the tree.
Hence, it is important to pay close attention to OSPF adjacency attributes; any mismatch could lead to unpredictable outcomes.