What is App-IDs, PCAPs and Custom Signatures?
Question
What is App-IDs, PCAPs and Custom Signatures?
Environment
-
Palo Alto Networks Next-Generation Firewalls
-
PAN-OS
-
App-ID
-
Custom Applications
-
Application Signatures
-
Wireshark or another packet analysis tool
Answer
What is App-ID?
Application Identification (App-ID) is a core technology of Palo Alto Networks Next-Generation Firewalls (NGFWs) that identifies applications traversing the firewall, regardless of the ports commonly associated with those applications.
Unlike traditional firewalls that primarily rely on IP addresses, protocols, and port numbers to control traffic, App-ID identifies applications using their unique characteristics and communication patterns.
App-ID uses multiple identification techniques, including:
-
Application Signatures: Identify applications by matching recognizable patterns within network traffic.
-
Application Protocol Decoders: Analyze supported protocols and provide application-specific contexts for identification.
-
Heuristics and Behavioral Analysis: Help identify applications based on their communication characteristics when signature-based identification alone is insufficient.
These mechanisms provide Layer 7 (L7) application visibility, allowing administrators to create application-based Security Policies and apply supported security inspection.
App-ID can also identify certain applications communicating over encrypted connections using information visible during the protocol negotiation. However, when the information required for identification is encrypted, an appropriate decryption policy may be necessary.
Application Dependencies and Parent Applications
Some applications rely on other applications or protocols to establish communication.
For example, an application communicating over HTTP may initially be identified as web-browsing before sufficient information becomes available to identify the specific application.
PAN-OS uses application dependencies to identify applications that require another application to function correctly.
For custom applications, administrators can also specify a parent application when appropriate. This allows the custom application to use the protocol decoding capabilities associated with the parent application.
Application dependencies and parent applications serve different purposes and should not be confused.
Although PAN-OS recognizes a large number of applications through its regularly updated Applications and Threats content, some proprietary, internally developed, or uncommon applications may not be recognized.
These applications may be classified as unknown-tcp or unknown-udp.
When a recognizable and sufficiently unique application pattern is available, administrators can create a custom App-ID signature to identify the traffic.
How Does App-ID Work?
When traffic traverses a Palo Alto Networks firewall, App-ID analyzes the communication to identify the application and enable application-based Security Policy enforcement.
Application identification is progressive and may change as additional packets are exchanged.
The general identification process is as follows:
-
Initial Session Processing: The firewall receives traffic and performs initial session and Security Policy processing based on the information available.
-
Application Signature Matching: App-ID examines available traffic characteristics and evaluates applicable application signatures.
-
Protocol Decoding: Supported protocol decoders interpret the application protocol and provide contexts for identifying applications within the traffic.
-
Heuristic Analysis: Additional identification techniques may be used when application signatures and protocol decoding alone are insufficient.
-
Application Identification: The firewall identifies the application based on the available evidence. Identification may become more specific as the session progresses.
-
Security Policy Enforcement: PAN-OS evaluates the identified application against the configured Security Policies and applies the relevant policy action and supported security inspection.
-
Logging and Visibility: Application information is available through the Traffic Logs and other monitoring features in PAN-OS.
Application Identification and Service Ports
App-ID is designed to identify applications independently of their commonly used ports. However, the service configured in the Security Policy determines which destination ports are permitted.
For applications communicating on their defined default ports, the recommended configuration is generally application-default.
If an application must communicate over a nonstandard port, administrators should review whether an explicitly defined service is required.
For custom applications, the appropriate default port can be configured within the custom application definition.
Application Identification and Encrypted Traffic
When traffic is encrypted using SSL/TLS or SSH, App-ID may still identify certain applications using visible protocol information and traffic characteristics.
However, if the application signature requires access to encrypted payload data, decryption may be necessary.
Without decryption, the firewall may identify the traffic only as a generic encrypted application, such as ssl or ssh, or as another application identifiable from the available information.
Note: Decryption is not mandatory for every App-ID identification. Its necessity depends on the application protocol, available metadata, and the information required for identification.
How Does PAN-OS Handle Unknown Applications?
During App-ID adoption or Security Policy optimization, administrators may encounter traffic that PAN-OS cannot identify.
Unknown applications should be investigated to determine whether they represent legitimate business applications, unsupported protocols, incomplete sessions, or potentially unwanted traffic.
PAN-OS commonly uses the following classifications:
| Classification | Description |
|---|---|
unknown-tcp | TCP traffic for which the application could not be identified. |
unknown-udp | UDP traffic for which the application could not be identified. |
incomplete | TCP traffic where a complete three-way handshake was not observed. |
insufficient-data | Traffic where insufficient application data was available to determine the application. |
Incomplete Sessions
The incomplete classification generally indicates that the firewall did not observe a complete TCP three-way handshake.
Possible causes include:
-
The server does not respond to the client's SYN.
-
The TCP handshake is interrupted.
-
Asymmetric routing prevents the firewall from observing the complete handshake.
-
The session terminates before the handshake completes.
For these sessions, investigate the network connectivity and TCP handshake before attempting to create a custom application signature.
Insufficient Data
The insufficient-data classification indicates that the firewall did not receive enough application data to identify the application.
For example, a TCP connection may complete successfully but exchange little or no application payload before terminating.
Possible causes include:
-
The application closes the connection before exchanging sufficient data.
-
The session contains limited application payload.
-
The application communicates using an uncommon or minimal protocol exchange.
Unknown TCP and UDP Applications
Traffic classified as unknown-tcp or unknown-udp may contain application data that PAN-OS cannot associate with an existing App-ID.
Possible reasons include:
-
Proprietary or internally developed applications.
-
Unsupported application protocols.
-
Application payloads that do not match existing signatures.
-
Uncommon protocol behavior.
-
Encrypted traffic with insufficient visible identification information.
In these scenarios, administrators should review the Traffic Logs and collect packet captures to determine whether a unique application signature can be created.
Should Application Override Be Used?
Application Override is not required to investigate unknown applications and should not be considered the default solution.
Application Override allows administrators to assign a predefined application classification to traffic matching specified criteria without normal App-ID identification.
However, traffic matching an Application Override policy bypasses normal App-ID processing, which can limit application-layer threat inspection.
When application identification and Layer 7 inspection are required, creating a custom application signature is generally preferred.
How to Investigate Unknown Applications
Before creating a custom App-ID signature, administrators should determine why the application is not being identified.
Step 1: Review the Traffic Logs
Navigate to:
Monitor > Logs > Traffic
Filter the relevant session using the source IP address, destination IP address, application, or port.
Review the following information:
-
Identified application
-
Source and destination addresses
-
Source and destination ports
-
Security Policy rule
-
Session end reason
-
Bytes and packets exchanged
-
Action taken by the firewall
Step 2: Verify the Connection
For TCP traffic, confirm that the three-way handshake completes successfully.
If the session is classified as incomplete, investigate connectivity or routing before proceeding with App-ID signature analysis.
Step 3: Verify Application Data
Determine whether the client and server exchange application payload.
If the TCP handshake completes but insufficient application data is exchanged, the firewall may classify the session as insufficient-data.
If application payload is present but the firewall cannot identify the protocol, further packet analysis may be required.
Step 4: Collect Packet Captures
Collect packet captures that include the initial connection establishment and the application data exchanged between the client and server.
Whenever possible, collect multiple sessions representing normal application operations.
These captures help identify application-specific patterns that can be used to create custom signatures.
How to Create a PCAP
A packet capture (PCAP) records network packets and allows administrators to analyze the communication between a client and server.
For App-ID investigations, a PCAP can help determine:
-
Whether the TCP handshake completes.
-
Whether application data is exchanged.
-
Which protocol is being used.
-
Whether the application payload is encrypted.
-
Whether a unique and consistent application pattern is present.
Collect a PCAP Using the PAN-OS Web Interface
-
Navigate to Monitor > Packet Capture.
-
Configure a packet filter for the relevant source and destination addresses.
-
Specify the protocol or port where necessary.
-
Enable filtering.
-
Configure the required packet capture stages.
-
Enable packet capture.
-
Reproduce the application traffic.
-
Disable packet capture.
-
Download the PCAP files for analysis.
The available packet capture stages include:
| Capture Stage | Description |
|---|---|
| Receive | Captures packets entering the firewall dataplane. |
| Firewall | Captures packets at the firewall processing stage. |
| Transmit | Captures packets transmitted by the firewall. |
| Drop | Captures packets dropped by the firewall. |
For additional information, refer to:
Note: Packet capture should be enabled only for the required troubleshooting period. Use restrictive filters to minimize resource consumption and avoid collecting unrelated traffic.
How to Analyze the PCAP
Once the PCAP has been collected, open it using Wireshark or another supported packet analysis tool.
Identify the relevant TCP or UDP conversation.
For TCP-based applications:
-
Locate the relevant application session.
-
Verify the TCP handshake.
-
Identify packets containing application data.
-
Right-click a relevant packet.
-
Select Follow > TCP Stream.
-
Examine the application data exchanged between the client and server.
For HTTP-based applications, review request methods, URI paths, headers, and other application-specific fields.
For proprietary TCP applications, look for recognizable byte sequences, protocol identifiers, message headers, or other consistent patterns.
The selected pattern should:
-
Be unique to the application.
-
Remain consistent across multiple sessions.
-
Be present during normal application communication.
-
Avoid dynamic values such as IP addresses, TCP sequence numbers, timestamps, or ephemeral ports.
-
Be sufficiently specific to minimize false-positive identification.
Note: More than one application session should be analyzed before selecting a signature pattern. Testing against unrelated application traffic is also recommended.
How to Create a Custom App-ID
Once a unique and consistent application pattern has been identified, administrators can create a custom application and associate a signature with it.
Step 1: Create the Custom Application
Navigate to:
Objects > Applications > Add
Configure the relevant application properties, including:
-
Application name
-
Category and subcategory
-
Technology
-
Risk level
-
Application characteristics
-
Parent application, if applicable
-
Default ports
-
Session timeout settings, where required
The application properties should accurately represent the behavior and intended use of the custom application.
Step 2: Create the Application Signature
Open the Signatures tab and select Add.
Configure the signature name, scope, and matching conditions.
Select the appropriate signature context based on the application protocol.
For example, an HTTP-based application may use an HTTP request URI context, while a proprietary TCP application may require a supported TCP payload context.
Configure a pattern that uniquely identifies the application.
Additional matching conditions may be used to improve accuracy and reduce false positives.
Important: Custom App-ID signatures support pattern matching using PAN-OS-supported regular-expression syntax. Hexadecimal notation may be used to represent binary data, but hexadecimal format is not mandatory for every signature.
The selected pattern must comply with the signature syntax and context requirements supported by the PAN-OS version.
Step 3: Commit the Configuration
Save the custom application and commit the configuration.
Step 4: Configure the Security Policy
Navigate to:
Policies > Security
Create or update the Security Policy to allow the custom application where required.
Configure:
-
Appropriate source and destination zones.
-
Required source and destination addresses.
-
Custom application in the Application field.
-
Appropriate service configuration.
-
Required security profiles.
-
Logging at session end.
When using application-default, ensure the custom application has the correct default port configuration.
Review application dependencies and any initial application classifications required for the session to reach the custom App-ID identification stage.
Step 5: Verify Application Identification
Generate new application traffic and navigate to:
Monitor > Logs > Traffic
Confirm that the firewall identifies the traffic using the newly created custom App-ID.
Verify that:
-
The correct application is identified.
-
The expected Security Policy is matched.
-
The application operates as expected.
-
Supported security inspection remains available.
-
Unrelated applications are not incorrectly identified.
If the custom application is not identified, review the signature pattern, selected context, parent application, decryption requirements, and application payload.
Best Practices
-
Investigate unknown applications before creating custom signatures.
-
Verify TCP connectivity before troubleshooting application identification.
-
Collect multiple PCAPs to identify consistent application characteristics.
-
Prefer protocol-specific signature contexts when appropriate.
-
Avoid matching dynamic packet fields or common protocol strings.
-
Use multiple signature conditions where necessary to improve accuracy.
-
Review decryption requirements for encrypted applications.
-
Avoid unnecessary Application Override policies.
-
Verify application dependencies and Security Policy configuration.
-
Validate custom signatures before deploying them in production.
-
Reevaluate custom signatures when application behavior changes.
See Also