QSFP28 Interface flaps on PA-5200 when connected to a Cisco Nexus 9000
3882891
Created On 05/23/19 01:41 AM - Last Modified 07/29/25 08:51 AM
Symptom
• When PA-5200 series firewall is connected to Cisco Nexus 9000 using QSFP28 (40g/100g) optics, flaps are observed on Palo Alto device while Nexus shows port as down.
• Loopback testing on the firewall by connecting two ports on the same firewall causes link to come up which rule out firewall issue.
• Link Settings are set to auto and supported optics were used. Example of port 22 state is shown below.
FW-secondary(passive)> show system state filter sys.s1.p22.phy
sys.s1.p22.phy: { 'link-partner': { }, 'media': QSFP-Plus-Fiber, 'sfp': { 'connector': MPO, 'encoding': 64B66B, 'identifier'
: QSFP28, 'transceiver': , 'vendor-name': INNOLIGHT , 'vendor-part-number': TR-FC85S-N00 , 'vendor-part-rev': 1ABh,
}, 'type': Ethernet, }
• CP brdagent.log displays the following output.
FW-secondary(passive)> less cp-log brdagent.log
cp brdagent.log 2019-05-17 15:08:20 vendor 'INNOLIGHT '; part 'TR-FC85S-N00 '; id 'QSFP28'
cp brdagent.log 2019-05-17 15:08:20 2019-05-17 15:08:20.850 -0400 Port 21: Fiber QSFP+ detected
cp brdagent.log 2019-05-17 15:08:20 2019-05-17 15:08:20.857 -0400 PORT21: board_port_sfp_nopop_0 -> board_port_startup, link: 1, mode: 0
cp brdagent.log 2019-05-17 15:08:20 2019-05-17 15:08:20.859 -0400 Disabling traffic TX & RX, BCM link 32
cp brdagent.log 2019-05-17 15:08:20 2019-05-17 15:08:20.864 -0400 PORT21: board_port_startup -> board_port_autoneg, link: 1, mode: 0
cp brdagent.log 2019-05-17 15:08:20 2019-05-17 15:08:20.873 -0400 gryphon_port_autoneg:1211 Get Mac status for port:32 en:1
cp brdagent.log 2019-05-17 15:08:20 2019-05-17 15:08:20.873 -0400 Disabling traffic TX & RX, BCM link 32
cp brdagent.log 2019-05-17 15:08:21 2019-05-17 15:08:21.358 -0400 QSFP+ detected on port 22
cp brdagent.log 2019-05-17 15:08:21 vendor 'INNOLIGHT '; part 'TR-FC85S-N00 '; id 'QSFP28'
cp brdagent.log 2019-05-17 15:08:21 2019-05-17 15:08:21.363 -0400 Port 22: Fiber QSFP+ detected
cp brdagent.log 2019-05-17 15:08:21 2019-05-17 15:08:21.363 -0400 PORT22: board_port_sfp_nopop_0 -> board_port_startup, link: 1, mode: 0
Environment
• PA 5200 series firewall connected to Cisco Nexus 9000 using supported qsfp28 optics.
• Cisco, release versions 7.0(3)I7(x). Specifically 7.0(3)I7(3) and 7.0(3)I7(5) only.
Cause
When ports are functioning at 40/100 gig link speed, Ports on some switches like Cisco Nexus 9000 may fail to re-initialize or link up when multiple link flaps are issued on the remote end.
Resolution
Remove and install the QSFP optic on Cisco Nexus 9000.
Additional Information
Flaps can happen on the following Nexus 9000 Switches:
- N9K-X9788TC-FX (Linecard) - Ports 49-52
- N9K-X9736C-FX (Linecard) - Ports 29-36
- N9K-C9364C (Top of Rack Nexus 9000) - Ports 49-64
- N9K-C9332C (Top of Rack Nexus 9000) - Ports 25-32
- N9K-C9336C-FX2 (Top of Rack Nexus 9000) - Ports 1-6, 33-36
- N9K-C93240YC-FX2-Z (Top of Rack Nexus 9000) - Ports 51-54
- N9K-C93240YC-FX2 (Top of Rack Nexus 9000) - Ports 51-54.
So far, the issue has been seen when port is operating at 40/100gig.