Palo Alto Firewall is unable to send logs to the Panorama or Log collector after PAN-OS upgrade to 10.2.
3880
Created On 11/20/23 02:54 AM - Last Modified 06/19/25 03:14 AM
Symptom
- PAN-OS upgraded of the Firewall to 10.2+
- >show logging-status on the Firewall shows that the connection to the Panorama/LC is inactive and no logs are being forwarded.
- Packet captures indicate that the tcp 3-way handshake completed with Client Hello being sent by the FW and received on the Panorama.
- However, the Server Hello sent by the Panorama is not received on the Firewall.
- Then the Firewall resets the connection.
Environment
- Panorama on PAN-OS 10.2+
- Firewall upgraded to PAN-OS 10.2+
- Firewall configured to send logs to Panorama
- Intermediate firewall between the Firewall and Panorama allowing application panorama on port 3978 in the security rule
Cause
- After PAN-OS upgrade to 10.2, the communication between the FW and Panorama starts using TLSv1.3.
- Therefore, this communication is being identified as "SSL/3978" on 10.2+ instead of "panorama/3978" as on PAN-OS 10.1 and earlier.
- The existing security rule on the intermediate PA-FW in the path allows application "panorama" and not application "SSL" on 3978.
- Thus, the intermediate FW blocks the traffic at the Server Hello.
Resolution
- On the intermediate Firewall, Change or configure the security rule to allow application "SSL" on port 3978.
- Commit the configuration.
Additional Information
Why is traffic on port 3978 Identified as SSL application instead of Panorama application?
Pcaps for reference:
FW pcaps :
13:58:09.954344 IP X.X.X.X.40698 > Y.Y.Y.Y.3978: Flags [S], seq 2850888901, win 29200, options [mss 1460,sackOK,TS val 1871328907 ecr 0,nop,wscale 7], length 0
13:58:09.954737 IP Y.Y.Y.Y.3978 > X.X.X.X.40698: Flags [S.], seq 3658760759, ack 2850888902, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 8], length 0
13:58:09.954760 IP X.X.X.X.40698 > Y.Y.Y.Y.3978: Flags [.], ack 1, win 229, length 0
13:58:09.955314 IP X.X.X.X.40698 > Y.Y.Y.Y.3978: Flags [P.], seq 1:387, ack 1, win 229, length 386
13:58:09.956098 IP Y.Y.Y.Y.3978 > X.X.X.X.40698: Flags [.], ack 387, win 119, length 0
13:58:09.958796 IP Y.Y.Y.Y.3978 > X.X.X.X.40698: Flags [R.], seq 1, ack 387, win 119, length 0
Panorama pcaps :
13:58:09.954601 IP X.X.X.X.40698 > Y.Y.Y.Y.3978: Flags [S], seq 2850888901, win 29200, options [mss 1460,sackOK,nop,nop,nop,nop,nop,nop,nop,nop,nop,nop,nop,wscale 7], length 0
13:58:09.954645 IP Y.Y.Y.Y.3978 > X.X.X.X.40698: Flags [S.], seq 3658760759, ack 2850888902, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 8], length 0
13:58:09.954812 IP X.X.X.X.40698 > Y.Y.Y.Y.3978: Flags [.], ack 1, win 229, length 0
13:58:09.955982 IP X.X.X.X.40698 > Y.Y.Y.Y.3978: Flags [P.], seq 1:387, ack 1, win 229, length 386
13:58:09.956012 IP Y.Y.Y.Y.3978 > X.X.X.X.40698: Flags [.], ack 387, win 119, length 0
13:58:09.958136 IP Y.Y.Y.Y.3978 > X.X.X.X.40698: Flags [P.], seq 1:2545, ack 387, win 119, length 2544. <<<<<<<<<< Server Hello not reaching the FW
13:58:09.958811 IP X.X.X.X.40698 > Y.Y.Y.Y.3978: Flags [R.], seq 387, ack 1461, win 119, length 0