Troubleshooting User-ID cache timeout

Troubleshooting User-ID cache timeout

26375
Created On 03/23/21 14:00 PM - Last Modified 06/12/23 13:58 PM


Symptom


Users have connectivity issues due to no longer matching security policies which are configured for specific user accounts. The traffic logs show the traffic was matching the correct policies at first and user info was being populated, however after some time the traffic started to hit wrong policies and no user info was populated.

Cause


This is likely due to the User-ID cache timeout, which is reaching the timeout value before a new IP to User Mapping is generated. This would cause that particular users to no longer have an IP to User Mapping on the firewall.

Resolution


When an IP to User Mapping is been generated, it comes with a timeout value, which is visible under Monitor Tab -> Logs -> User ID on the webUI.

User-added image

This timeout dictates how long the mapping will be stored in cache until it is removed. In addition it is refreshed if a new User-ID event processed. You can view the current TTL of IP to User mapping entries by using these CLI commands: 
show user ip-user-mapping all
show user ip-user-mapping ip <ip>

Please find below some sample outputs:
   User-added image

To trace the issue down to the cache timer:

1. Two logs need to be filtered by the user's source IP:
  • Monitor Tab > Logs > Traffic
  • Monitor Tab > Logs > User ID
2. In the traffic logs, find the first entry where the user started to hit the unintended rule. In most environments this would be seen as a policy-deny log entry for the interzone-default rule, or a manually configured catch-all deny rule. You would also see this as the first entry where the Source User field starts to be blank.

3. Note the time the log was generated, and then move onto the User ID logs. 
  • Find the last entry before issue occurred for that user's IP address
  • Note the time of that entry and add the timeout for that entry to it. 
4. Now compare the result of that to the time of the traffic log which was noted. If the result is earlier than the traffic log's time, it shows that the IP to User mapping timed out as expected and the cache timeout needs to be adjusted.

User-added image

For example:
  • In the traffic log, the first entry to have a blank Source User was 03/23 06:37:19.
  • In the User ID log, you see an entry at 03/23 06:32:18 with a timeout of 300 (5 minutes).
06:32:18 + 5 mins = 06:37:18 should be the expected time that the User to IP Mapping cache TTL reached 0 and was removed. This is 1 second before the Source User was showing blank in the logs which shows that the User to IP mapping must have expired.
 
In conclusion:

The cause of the issue in this case is that events which generate an IP to User Mapping happen less frequently than the cache timeout.
Other users also viewed:
Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u0000001Uu5CAE&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail

Choose Language