Radius MFA (Push Notification) authentication is failing due to timeout
20325
Created On 04/24/20 14:56 PM - Last Modified 05/01/20 03:05 AM
Symptom
- Authentication requests are timing out on the firewall
- All authentication requests are seen failing in "authd" logs with the following error message.
grep mp-log authd.log pattern auth_svr_timeout_sent_request
2020-04-24 12:53:51.205 +0300 debug: auth_svr_timeout_sent_request(pan_auth_svr.c:263):timeout auth
request (authd id=6723213794060324594, username=test) since total elapsed sec 25 >= max allowed secs: 25
- Radius server is not receiving any requests from the firewall
- Sessions on the firewall seen in discard state between the firewall and radius server
show session all filter source <IP address on the firewall> destination <radius server IP>
show session all filter source <radius server IP> destination <IP address on the firewall>
Environment
- Any PAN-OS.
- Palo Alto Firewall.
- Radius server is reachable through one of the dataplane interfaces or through another firewall
Cause
- The default Radius application session timeout is 30 seconds
- If it takes the Radius server more than 30 seconds to respond back with the Access-Accept Message, then the session on the firewall would timeout
- Since the session times out, the response from the server would not match an existing session and since it is destined towards a random port number, it gets dropped on the firewall and the session is placed in discard state
- As the firewall tries sending additional requests towards the server, this discard session will remain active as its TTL of 60 seconds keep getting refreshed by the requests ( Example session-id shown below)
admin@PA-VM-8> show session id 727877
Session 727877
c2s flow:
source: 192.168.1.10 [trust]
dst: 192.168.2.10
proto: 17
sport: 55715 dport: 1812
state: DISCARD type: FLOW
src user: unknown
dst user: unknown
s2c flow:
source: 192.168.2.10 [trust]
dst: 192.168.1.10
proto: 17
sport: 1812 dport: 55715
state: DISCARD type: FLOW
src user: unknown
dst user: unknown
start time : Fri Apr 24 17:35:49 2020
timeout : 60 sec
time to live : 45 sec
total byte count(c2s) : 62
total byte count(s2c) : 3302
layer7 packet count(c2s) : 1
layer7 packet count(s2c) : 28
vsys : vsys1
application : unknown-udp (insufficient)
rule : intrazone-default
service timeout override(index) : False
session to be logged at end : False
session in session ager : True
session updated by HA peer : False
address/port translation : source
layer7 processing : enabled
ctd version : 10
URL filtering enabled : False
session via syn-cookies : False
session terminated on host : True
session traverses tunnel : False
session terminate tunnel : False
captive portal session : False
ingress interface : ethernet1/5
egress interface : ethernet1/5
session QoS rule : N/A (class 4)
tracker stage firewall : host service
Resolution
- Clear the session that is in discard state using the "clear session id <session id number>" command
- Increase the default radius application timeout from 30 seconds to a higher value based on how long it might take the server to respond or the user to accept the push notification (120-150 seconds is a reasonable value)
- Configure a custom service object with a desired timeout value and apply it to the radius security policy on the firewall
Additional Information
Note: For the impact of any debug commands on the Firewall Refer to this article.