Representation of the URL Logs in a SIEM Solution
Symptom
The URL logs in the Palo Alto Networks devices are exported as a part of the threat logs. As a part of the threat logs they are shown in security information and event management (SIEM) solutions.
Resolution
From the SIEM the threat logs can be filtered and reported on if needed. See the example below, if using Splunk as a SIEM and if looking at the logs using the filter: url dst_hostname="www.google.nl":
Shown below after the type of log is THREAT, there is another token which represents the subtype: url:
If exporting one of those events in full it will show the following:
Aug 5 14:56:46 Ilija-PA-VM-2.al.com 1,2014/08/05 14:56:46,007200001619,THREAT,url,1,2014/08/05 14:56:40,192.168.8.89,173.194.41.175,10.193.17.8,173.194.41.175,allow_all,,,ssl,vsys1,Trust-L3,Untrust-L3,ethernet1/2,ethernet1/1,forward to panorama and splunk,2014/08/05 14:56:46,196863,1,55143,443,52716,443,0x408000,tcp,alert,"www.google.nl/",(9999),search-engines,informational,client-to-server,67483,0x0,192.168.0.0-192.168.255.255,US,0,,0,,
This event is a Threat event, but the subtype is a URL. In the event above, see the parameter "url" just after the THREAT. Filtering in the SIEM solution based on this subtype will result in the URL logs being displayed. The action, rule, category and other associated information will also be displayed.
If a different representation is needed for a specific SIEM solution, the predefined tokens "$subtype|$type" can be used:
As an example we might use this output in the Custom Log Format (Under Device > Server Profiles > Syslog):
<threat>CEF:0|Palo Alto Networks|PAN-OS|5.0|$subtype $threatid|$type|$number-of-severity|rt=$cef-formatted-receive_time deviceExternalId=$serial src=$src dst=$dst sourceTranslatedAddress=$natsrc destinationTranslatedAddress=$natdst cs1Label=Rule cs1=$rule suser=$srcuser duser=$dstuser app=$app cs3Label=Virtual System cs3=$vsys cs4Label=Source Zone cs4=$from cs5Label=Destination Zone cs5=$to deviceInboundInterface=$inbound_if deviceOutboundInterface=$outbound_if cs6Label=LogProfile cs6=$logset cn1Label=SessionID cn1=$sessionid cnt=$repeatcnt spt=$sport dpt=$dport sourceTranslatedPort=$natsport destinationTranslatedPort=$natdport flexString1Label=Flags flexString1=$flags proto=$proto act=$action msg=$misc cs2Label=URL Category cs2=$category flexString2Label=Direction flexString2=$direction externalId=$seqno requestContext=$contenttype</threat>
After that the URL logs can also have the custom fields that can be in a uniformed format for the SIEM administrator to create reports.
With the usage of the above custom log formatting we get the following log in the SIEM:
Oct 10 14:24:00 CEF:0|Palo Alto Networks|PAN-OS|5.0|url (9999)|THREAT|1|rt=Oct 10 2014 12:24:00 GMT deviceExternalId=007200001618 src=172.16.100.89 dst=74.125.230.229 sourceTranslatedAddress=10.193.91.100 destinationTranslatedAddress=74.125.230.229 cs1Label=Rule cs1=allow_corp_services suser=al\iladmin duser= app=ssl cs3Label=Virtual System cs3=vsys1 cs4Label=Source Zone cs4=Trust_L3 cs5Label=Destination Zone cs5=Untrust_L3 deviceInboundInterface=ethernet1/2 deviceOutboundInterface=ethernet1/1 cs6Label=LogProfile cs6=Forward_to_PANORAMA_and_Splunk cn1Label=SessionID cn1=211699 cnt=1 spt=20949 dpt=443 sourceTranslatedPort=54006 destinationTranslatedPort=443 flexString1Label=Flags flexString1=0x408000 proto=tcp act=alert msg="clients1.google.com/" cs2Label=URL Category cs2=search-engines flexString2Label=Direction flexString2=client-to-server externalId=982843 requestContext=