Some URLs are not showing in URL filtering log

• Some URLs are not showing in URL filtering logs when a category has been changed from allow to alert, block.
• We see that Firewall receives the URL: registry-1[.]docker[.]io/v2/

• The URL filtering logs does show the URL the forward slash at the end "/" is removed. Example: registry-1[.]docker[.]io/v2

• Palo Alto Networks Firewalls.
• Supported PAN-OS
• URL Filtering
• Content-ID

• The cause of this behavior consists of two parts; having Log container page only enabled on the URL filtering settings and missing the URL content type under Content-ID Features
• These are the default URL Content types supported by Palo Alto Networks firewalls. They specify the types of URLs that the firewall tracks or logs based on content type
  
• The "Log container page only" feature is checked by default under Objects > Security Profile > URL Filtering > URL Filtering object > URL Filtering settings 
  
  
  
• With "Log Container Page Only" unchecked, the URL can be seen exactly which sub-resource on a URL triggered alerts.
• For example, if a user visits a site like cnn.com, the admin can see a URL log flagged as high-risk or even a threat.
• The admin can pinpoint the specific resource and determine whether it was an ad, a tracking script, or an embedded malicious JavaScript that caused the alert.
• With "Log Container Page Only" checked, when there is an infected host in an environment and the Firewall admin is researching the root cause, the admin will start looking through the URL Filtering logs and see many alerts and threats in the URL filtering logs.
• However, the admin can not find what the user clicked on or which sub-resource the endpoint interacted with on the page. Evidence of a website URL might suffice, but some organizations might need additional details to determine the root cause. 

1. Uncheck the "Log Container Page Only" under Objects > Security Profile > URL Filtering > URL Filtering object > URL Filtering settings
2. Commit the changes.

In case your organization requires to leave the Log container page only setting enabled, follow the below process instead:

1. Perform curl -i to compare the URL Content Type of the URL from the working and non-working scenario. Non-working entry with trailing slash ('/') is classified as application/json
   
2.  Navigate to Device > Setup > Content-ID > Content-ID Features > Container Pages and create a custom list of content types. The list should contain all the default content types as well as the newly specified ones:
   
3. Commit the changes
   
   

• Ultimately, having the feature checked or unchecked depends on the organization's needs and how deep the admin needs to go into logs.
• Reducing log volume is great for simplicity, but it can reduce the ability to chase down vulnerabilities or meet detailed compliance requirements.
• If the organization lacks additional security layers like a secure browser, proxy, or endpoint detection, then it will be a consideration to have the feature unchecked to retain more visibility.
• Container pages are set per virtual system, which you select from the Location drop-down. If a virtual system does not have an explicit container page defined, the firewall uses the default content types.
• Adding new content types for a virtual system overrides the default list of content types. If there are no content types associated with a virtual system, the default list of content types is used.
• Log Container Page Only - impact?
• See Content-ID help page for more information