Video Tutorial: Advanced URL Filtering

 

Video Transcript:
This is a Palo Alto Networks Video Tutorial. My name is Joe Delio and I am a Solutions Engineer from the Palo Alto Networks Community team. In today's Video Tutorial I will be talking about "Advanced URL Filtering."

This is the second video of a series talking about URL Filtering. This video is designed to help you better understand and configure URL filtering on PAN-OS 6.1. Since we already covered the basic options and components of URL filtering in the previous video, we can start to cover some more advanced topics.

We will be covering the following topics in this Advanced URL Filtering Video Tutorial:

1. Monitoring Web Activity

• Use the ACC to Monitor Web Activity
• View URL Filtering Reports
• Reports - User Activity, URL Filtering

2. Response Pages
3. Match traffic based on URL category for policy enforcement.
4. Decryption
5. SafeSearch

1. First item. Use the ACC to Monitor Web Activity
In my previous video on URL filtering, I showed you how to create a passive URL filtering policy, as well as how to use the URL filtering logs to see what sites are being accessed. 

** Now I will show you how to use the ACC (Application Command Center) to Monitor Web Activity.
For a quick view of the most common categories being accessed in your environment, select the ACC tab and scroll down to the URL Filtering section. Along the top of this window, you can also set the time range, sort by option, and define how many results will appear. Here you will see the most popular categories that are accessed by your users sorted by the most popular at the top of the list.

** Next up are all of the URL reports.
The first report that I will show you is the URL Filtering reports. To view the default URL filtering reports, select Monitor > Reports and under the URL Filtering Reports section, choose one of the reports. You can generate reports on URL Categories, URL users, Websites accessed, Blocked Categories, and more. The reports are based on a 24 hour period and the day is selected by choosing a day in the calendar section. You can also export the report to PDF, CSV, or XML.

** Next is the User Activity Report.
To configure the User Activity Report, you have to Select Monitor > PDF Reports > User Activity Report.

• Click 'Add' in the lower left hand of the screen.
• Enter a report Name and select the report type either User or Group. I selected user for 1 user.
• Enter the Username/IP address for a user report or enter the group name for a user group report.
• I am testing with my machine IP 172.16.77.209.

NOTE: You must enable User-ID in order to be able to select user or group names. If User-ID is not configured, you can select the type User and enter the IP address of the user’s computer.

• Select the time period. You can select an existing time period, or select Custom. I am doing 7 days.
• Select the Include Detailed Browsing check box, so browsing information is included in the report.
• To run the user activity report and then download the report. Click the Run Now button.

After the report is generated, click the Download User Activity Report link. View the user activity report by opening the PDF file that was downloaded. The top of the report will contain a table of contents.

Click an item in the table of contents to view details. For example, click Traffic Summary by URL Category to view statistics for the selected user or group.

** Next is the Custom URL Filtering Reports
To generate a detailed report that can also be scheduled, you can configure a custom report and select from a list of all available URL filtering log fields.

Step 1 - Add a new custom report.

1. Select Monitor > Manage Custom Reports and click Add.
2. Enter a report name in the Name field. For example, "My-URL-Custom-Report."
3. From the Database drop-down, select URL Log.

Step 2 - Configure report options.

1. 1Select the Time Frame drop-down and select a range.
2. (Optional) To customize how the report is sorted and grouped, select Sort By and chose the number of items to display (top 25 for example) and then select Group By and select an option such as Category, and then select how many groups will be defined.
3. In the Available Columns list, select the fields to include the report. The following columns are typically used for a URL report:

• Action
• Repeat Count
• Category
• Destination Country
• Source User or Source IP
• URL

Step 3 - Run the report to check the results. If the results are satisfactory, set a schedule to run the report automatically.

1. Click the Run Now icon to immediately generate the report that will appear in a new tab.
2. (Optional) Click the Schedule check box to run the report once per day. This will generate a daily report that details web activity over the last 24 hours. To access the report, select Monitor > Report and then expand Custom Reports on the right column and select the report.

Step 4 - Save the configuration by committing.

2. Response Pages
Let's talk about response pages. These are HTML web pages that are used in conjunction with URL Filtering, QoS and Decryption. But we are only going to talk about the URL Filtering response pages today. They are located under Device > Response Pages.

The firewall provides three predefined response pages that display by default when a user attempts to browse to a site in a category that is configured with one of the block actions in the URL Filtering Profile (block, continue, or override) or when Safe Search Enforcement is enabled:

• URL Filtering and Category Match Block Page — Access blocked by a URL filtering profile or because the URL category is blocked by a security policy.
• URL Filtering Continue and Override Page — Page with initial block policy that allows users to bypass the block. With the override page, after clicking Continue, the user must supply a password to override the policy that blocks the URL.
• URL Filtering Safe Search Block Page — Access blocked by a security policy with a URL filtering profile that has the Safe Search Enforcement option enabled. The user will see this page if a search is performed using Google, Bing, Yahoo, or Yandex and their browser or search engine account setting for Safe Search is not set to strict.

You can either use the predefined pages, or you can Customize the URL Filtering Response Pages to communicate your specific acceptable use policies and/or corporate branding. In addition, you can use the URL Filtering Response Page Variables for substitution at the time of the block event or add one of the supported Response Page References to external images, sounds, or style sheets. All of the details are covered in the Admin Guides. The links are at the end and in the transcript below.

Please note:
You must enable the 'Response Pages' option inside of the Management Profile.
This is found Network > Network Profiles > Interface Management > profile.

This must be enabled and then applied to an interface that faces the customer. In this example I have the 'allow-secure' profile that has response pages enabled.  You can see when I go into
Network > Interfaces and select ethernet1/4, which is my 'trust' interface that faces my clients. I go into Advanced and under 'Other Info' I have 'allow-secrure' selected.

For more information on URL Filtering response pages and variables that can be used, please see the admin guide for your version of PAN-OS. Links are available at the end of this transcript.

3. Match traffic based on URL category for policy enforcement.
Next it is to Match traffic based on URL category for policy enforcement.

After you have monitored URL traffic and ran through the steps I just outlined, you should have a basic understanding of what types of websites and website categories your users are accessing. With this information, you are now ready to create custom URL filtering profiles and attach them to the security policy rule(s) that allow web access.

Now we will create a simple policy that matches traffic based upon URL Category to control access to Facebook.com and block all other social media sites.

In this use case, a URL filtering policy is applied to the security policy that allows web access for your users, block the social-networking URL category, but the allow list in the URL profile is configured to allow the social networking site Facebook.

The first thing that we have to do is create a new URL filtering profile. This is done inside of Objects > Security Profiles > URL Filtering.

Let's start off by cloning the default policy to customize.
Highlight the 'default' profile, and then "clone" it at the bottom of the screen.
Scroll down to the bottom of the screen and then click on 'default-1' to modify it.
We wanted to Allow facebook, but block other Social Media sites. Lets start off by renaming this to "allow-facebook," then inside of the allow list we are going to add 2 entries in the Allow List for facebook. facebook.com and *.facebook.com. This is very important that you add both URLs because they are not the same.

Note: You may want to keep that in mind when having to block or allow any site in the future.

Next, lets find 'social-networking' inside of the URL Category.

You can do this by either scrolling down the list for it, but since there are more than 60 URL Categories, you can save some time by typing in 'social' in the search window and pressing enter.

Click on the action allow and change it to 'block'.
Then click OK.

Next we need to place this profile into a rule.
Inside of Policies > Security and find the rule that allows Internet access to the web.

My rule is 'trust-to-untrust.'
Click on it's name, and then click on the 'Actions' tab.
Then under profile setting, change it to the new URL profile we just made - 'allow-facebook,' then click OK.
Then commit this config.

Now, looking from the client accessing the internet, we see what happens when you access twitter.
Notice the block page?
Then go to facebook, and notice how you can see the page, and will be able to log in.

Looking at the logs now, inside Monitor > URL Filtering.
Here is what the logs will look like when traffic is getting denied. You do not see the allow in the URL Logs, as it is allowed now.
URL traffic will only show up in the URL logs if you choose to alert or block.  You will have to look for access to the site in the traffic logs if you want to see the allowed traffic.
NOTE: Please remember that in order to fully identify and control sites that use SSL, Decryption needs to be enabled on the firewall.

4. Decryption
Which that now brings me to the 4th item, Decryption. URL categories can also be used as match criteria in a decryption policy.
I will not cover everything for Decryption, just about how URL Filtering is used in conjunction with Decryption.

URL categories will be used in decryption policies to control which web categories should be decrypted or not decrypted.

You control this inside of Policies > Decryption.
You can see here the first rule is a no-decrypt rule that will not decrypt user traffic if the website category is financial-services or health-and-medicine and the second rule will decrypt all other traffic.

The decryption policy type is ssl-forward-proxy, which is used for controlling decryption for all outbound connections performed by users.
Note: Please see the Admin guides, the Knowledge Base or future Video tutorials on how to configure SSL Forward Proxy Decryption.

5. SafeSearch
Now onto the last section, which is 'Safesearch.'

Just about every popular search engine - Yahoo, Google, Bing and Youtube have a 'Safesearch' option to filter out NSFW or Adult results from search results. When this option is enabled on the Palo Alto Networks firewall, this will prevent users who are searching the Internet using one of the following search providers — Bing, Google, Yahoo, Yandex, or YouTube—from viewing the search results unless the strictest safe search option is set in their browsers for these search engines.

By default, when you enable safe search enforcement, when a user attempts to perform a search without using the strictest safe search settings, the firewall will block the search query results and display the URL Filtering Safe Search Block Page. This page provides a link to the search settings page for the corresponding search provider so that the end user can enable the safe search settings. If you plan to use this default method for enforcing safe search, you should communicate the policy to your end users prior to deploying the policy.

See the admin guide under Search Provider Safe Search Settings for details on how each search provider implements safe search.

Step 1. Enable Safe Search Enforcement in the URL Filtering profile.

1. Select Objects > Security Profiles > URL Filtering.
2. Select an existing profile to modify, or clone the default profile to create a new profile.
3. On the Settings tab, select the Safe Search Enforcement check box to enable it.
4. Make sure that profile is used in the security policy rule that allows traffic from clients in the trust zone to the Internet.
5. Commit this to take effect.

What you need to be aware of with Safesearch:

1. You can restrict users to specific search engines if you have the need. All of the details on how to do this are covered in the admin guides.
2.  Because most search engines encrypt their search results, you must enable SSL forward proxy decryption so that the firewall can inspect the search traffic and detect the safe search settings.
3. You can also Enable Transparent Safe Search Enforcement. Please view the admin guide for more information.

If you have any questions about URL filtering, please make sure you watch my first URL Filtering video below.
That concludes this video tutorial. We hoped you enjoyed this video. Thanks for watching.