URL Filtering - Dynamic Block List - External Block List EDL
Symptom
In earlier versions of PAN-OS, Dynamic Block List (EDL - External Dynamic List) or External Block Lists (EBL) allowed a firewall administrator to block a list of IP subnets or ranges based on an external file containing the IPs.
Starting with PAN-OS 7.1, blocking like this has become easier than ever with the introduction of URLs as a separate list type.
Environment
- PAN-OS 7.1
- External Dynamic List (EDL)
Resolution
Requirements
Each URL list is treated as a category, using the name of the list as the category name.
- Those categories are available in URL filtering profiles and in the security rules.
- Updates can be set to 5 minutes, hourly, daily, weekly, or monthly.
- If a 5-minute interval is set, only changes to list content will trigger a commit, and only once-per-hour.
- If the list is updated on the external site, but it’s not seen on the local firewall, check the config audit/candidate config to see the new items pulled from the list.
- The URL list can be hosted on an HTTPS site. All validation will be checked (CA validation, CN/SAN check, expiration check, OCSP, & CRL). Note that this validation is only supported in PAN-OS 8.0 and later (https://www.paloaltonetworks.com/documentation/80/pan-os/newfeaturesguide/authentication-features/authentication-for-external-dynamic-lists).
PA-200, PA-500, PA-2000, PA-3000, PA-4000 and PA-VM platforms
- 30 lists combined (IP + DNS + URL).
- 50,000 IPs total with no individual list limitation.
- 50,000 total DNS + URLs combined, no limit per list.
PA-5000 & PA-7000 series
- 30 lists combined (IP + DNS + URL).
- 150,000 IPs total with no individual list limitation.
- 50,000 total DNS + URLs combined, no limit per list.
Note: If more than the maximum 50K URLs is used, the firewall will use the first 50K and truncate the list. A system log is generated for this event.
Configuration
Step 1. To create a new External list, navigate to Objects > External Dynamic Lists > Add. I used 'Bad Mojo' as the name. Add the external Source. I used "http://www.example.com/url-list.txt". Also notice the 'repeat.' which is set to 'Five Minute' as the refresh rate for this external list.
Step 2. To create a New URL Filtering Profile inside Objects > Security Profiles > URL Filtering > Add a new profile. Scroll to the bottom to see the newly created list.
Note: Action is 'allow' for new profiles created after the EDL is created.
Step 3. To edit an existing profile, choose Objects > Security Profiles > URL Filtering, Edit it by clicking on the name.
Note: Action is 'none' until an admin changes it. Same behavior as custom URL categories.
Step 4. Inside a Secutiy Policy View (Policies > Security), click on a rule name to edit the rule, then inside the Service/URL Category, you will see the Bad Mojo list under External Dynamic Lists:
Step 5. Commit to enable this list.
List format requirements
- List must be a plain text document (no HTML, no PDF, etc.).
- Scheme is optional, and will be truncated if found – even if it is incomplete.
- http:// is not needed.
- Wildcards (*) are supported.
- *.example.com/hacked/*
- www.example.net/wp-*/debug/
- Maximum length per line is 1024 characters.
- Double-byte characters not supported.
- If specifying a domain, use both formats (as with custom URL categories):
- example.com
- *.example.com
Multi-vsys environment:
> set shared/<vsys vsys> external-list <tab>
{displays a list of current added lists}
<name>
> set shared/<vsys vsys1> external-list <name>
+ description description
+ url url
+ type type
> recurring recurring
<Enter> Finish input
> set shared/<vsys vsys1> external-list <name> type <tab>
+ domain Domain List
+ ip IP List
+ url URL List
Single-vsys environment
> set external-list <tab> -list of current added lists <name> > set external-list <name> + description description + url url + type type > recurring recurring <Enter> Finish input > set external-list <name> type <tab> + domain Domain List + ip IP List + url URL List
Panorama:
> set shared/<device-group dg name> external-list <tab>
{displays a list of current added lists}
<name>
> set shared/device-group dg name> external-list <name>
+ description description
+ url url
+ type type
+ recurring recurring
<Enter> Finish input
> set shared/device-group dg name> external-list <name> type <tab>
+ domain Domain List
+ ip IP List
+ url URL List
CLI changes (refresh & show commands)
> request system external-list show type
+ domain Domain list type
+ ip IP list type
+ url URL list type
> request system external-list show type url name <tab>
+ edl-url1 edl-url1
+ edl-url2 edl-url2
+ <name> <name>
> request system external-list show type url name edl-url1
{displays list of URL entries}
> request system external-list refresh type
+ domain Domain list type
+ ip IP list type
+ url URL list type
> request system external-list refresh type url name <tab>
+ edl-url1 edl-url1
+ edl-url2 edl-url2
+ <name> <name>
> request system external-list refresh type url name edl-url1
Panorama
When managing versions older than 7.1, only 'IP' type external block lists may be used.
Objects of type 'url' will be stripped from the config when pushed to a 7.0 or older PAN-OS version.
- If a policy references a URL list type, commit will fail.