How does the URL Filtering Inline ML feature work?

Why is the URL in-line ML verdict always unknown?

PAN-OS >= 10.0.
Active PAN-DB URL Filtering license.

The URL Filtering Inline ML feature was added to PanOS in software version 10.0 and it works with URL Filtering (PanDB) to inspect URLs that match specific criteria. 

When an HTTP request is sent from a client, the URL Filtering profile inspects the traffic, and the action associated with the request is taken. When the server replies to the request, the MLAV configuration, when enabled and the specific criteria are met, is used to inspect the reply and the corresponding MLAV action affects this portion of the session as example1 and example2 show.

This is the reason for the URL log entry which is then followed by the MLAV action entry in the URL Filtering log. Also, MLAV doesn't have a designated block page. So, for the MLAV block action, you will see either a reset page or a partial page that doesn't have malicious content.

Example1:

Example2:

 

If the URL filtering profile (either the normal PanDB category or custom/predefined URL category) blocks the URL in the HTTP request, the firewall never sees the response packet from the server and the MLAV inspection never takes place.

The firewall will not scan all URLs against MLAV cloud. The firewall does some “Pre-filtering” that helps to avoid unnecessary lookups. PAN-DB has a specific MLAV flag response for those types of records and as such, the firewall will not run the inline ML feature in those instances. The result of this interaction between the inline ML inspection and PanDBs MLAV flag response yields a verdict of “Unknown” in the Inline ML analysis.

MLAV works in addition to PAN-DB, meaning that it does not focus on all URLs inspected. The criteria for MLAV to inspect an URL are as follows.

• Leverage App-ID to only inspect web traffic (web-browsing)
• Exclude known malicious content, (PAN-DB or MLAV), and well-known sites with Low risk
• Include Unknown sites and sites that are Benign that have been previously categorized with Medium or High risk

A session must identify the “web-browsing” application and a URL must either be categorized as "Unknown" in PAN-DB or have a benign category with a risk level of "Medium or High” for MLAV to then take action and inspect the URL in real-time.

If you configure the url filtering inline ML feature and you see the URL log entry where the "Unknown" verdict only is seen in "INLINE ML VERDICT" ( not followed by "phishing" or "malicious-javascript" ) as Example3 shows,MLAV hasn't found anything suspicious and malicious in the site and the site hasn't been marked as phishing or malicious-javascript by MLAV. These include sites with known malicious content (sites categorized as such in PAN-DB) and sites that are categorized with one of the "Benign" categories and that also have a "Low" risk categorization.

Example3: