URL filtering lookup and enforcement differences between clear text HTTP and encrypted HTTPS traffic
27764
Created On 08/12/20 22:08 PM - Last Modified 08/12/20 22:11 PM
Objective
This KB article describes how URL filtering policy is applied during HTTP session setup and what the behavior of the firewall is during various clear text and encrypted scenarios.
URL filtering policy allows network and web security administrators to control access to websites by URL category, and is used to enforce web acceptable use policy (AUP) as well as to perform critical security functions such as blocking access to known malicious URLs, including malware, phishing, command-and-control, and other high risk websites. The specifics of what data is used to perform URL category lookup, and how policy enforcement impacts traffic flow depends on several factors and is described in more detail below.
Procedure
Clear text HTTP web transactions
When a web transaction is performed over clear text HTTP (typically port 80), the URL filtering policy inspects the HTTP Host and URL path headers in the client request to perform enforcement. The firewall can be configured to lookup and enforce policy before the initial request packet egresses the firewall to the web, or for increased performance, the lookup and enforcement can happen after the initial request packet is sent, where subsequent packets are blocked if the policy lookup results in a block action. (More information on this setting and other configuration parameters of URL filtering can be found in the PAN-OS Admin Guide, available here: https://docs.paloaltonetworks.com/pan-os/10-0/pan-os-admin/url-filtering/configure-url-filtering.html)
Encrypted HTTPS web transactions
If a web transaction is performed over encrypted HTTPS (typically port 443), the URL filtering policy inspects the Server Name Indication (SNI) field within the TLS Client Hello handshake. The firewall can be configured to lookup and enforce policy before the initial request packet egresses the firewall to the web, or for increased performance, the lookup and enforcement can happen after the initial request packet is sent, where subsequent packets are blocked if the policy lookup results in a block action.
Decrypted HTTPS web transactions
If a web transaction is performed over encrypted HTTPS, and the administrator has configured a decryption rule that decrypts the web transaction, the URL filtering policy inspects the HTTP Host and URL path headers in the client request but does not consider the SNI field within the TLS Client Hello handshake. This is done because the combined Host and URL path in the HTTP request header provides more granular information that can be used to perform a more specific URL category lookup for enforcement.
Note: While this results in a more accurate and specific URL category match, it results in the egress of the TLS Client Hello message being transmitted before URL filtering inspection can be performed on the subsequent HTTP request sent from the client. This presents a possible risk of bidirectional data channel between a compromised host within a network and a malicious web server on the Internet by using fields within the TLS handshake process to exchange data. Customers concerned with this behavior for decrypted HTTPS sessions are advised to pursue either of the following two options:
- Palo Alto Networks is currently working on a PAN-OS software update to address this behavior by adding a URL filtering policy check on both the TLS SNI field and the HTTP Host and URL headers for decrypted HTTPS transactions. The PSIRT advisory related to this issue (CVE-2020-2035) will be updated when a software update is available.
- For customers that wish to immediately mitigate the risk described above can view the KB article How to enforce URL filtering policy on TLS handshakes for decrypted HTTPS sessions for workaround options to evaluate and deploy if necessary.
Note: That this issue only affects HTTPS traffic that is decrypted using HTTPS Forward Proxy decryption policy, and can only be leveraged by an attacker that has already compromised a host inside the network and seeks to use this specific technique to perform command-and-control.