What is App-IDs, PCAPs and Custom Signatures?

What is App-IDs, PCAPs and Custom Signatures?

129129
Created On 09/26/18 13:47 PM - Last Modified 10/28/26 16:43 PM


Question


What is App-IDs, PCAPs and Custom Signatures?

Environment


  • Palo Alto Networks Next-Generation Firewalls

  • PAN-OS

  • App-ID

  • Custom Applications

  • Application Signatures

  • Wireshark or another packet analysis tool



Answer


What is App-ID?

Application Identification (App-ID) is a core technology of Palo Alto Networks Next-Generation Firewalls (NGFWs) that identifies applications traversing the firewall, regardless of the ports commonly associated with those applications.

Unlike traditional firewalls that primarily rely on IP addresses, protocols, and port numbers to control traffic, App-ID identifies applications using their unique characteristics and communication patterns.

App-ID uses multiple identification techniques, including:

  • Application Signatures: Identify applications by matching recognizable patterns within network traffic.

  • Application Protocol Decoders: Analyze supported protocols and provide application-specific contexts for identification.

  • Heuristics and Behavioral Analysis: Help identify applications based on their communication characteristics when signature-based identification alone is insufficient.

These mechanisms provide Layer 7 (L7) application visibility, allowing administrators to create application-based Security Policies and apply supported security inspection.

App-ID can also identify certain applications communicating over encrypted connections using information visible during the protocol negotiation. However, when the information required for identification is encrypted, an appropriate decryption policy may be necessary.

Application Dependencies and Parent Applications

Some applications rely on other applications or protocols to establish communication.

For example, an application communicating over HTTP may initially be identified as web-browsing before sufficient information becomes available to identify the specific application.

PAN-OS uses application dependencies to identify applications that require another application to function correctly.

For custom applications, administrators can also specify a parent application when appropriate. This allows the custom application to use the protocol decoding capabilities associated with the parent application.

Application dependencies and parent applications serve different purposes and should not be confused.

Although PAN-OS recognizes a large number of applications through its regularly updated Applications and Threats content, some proprietary, internally developed, or uncommon applications may not be recognized.

These applications may be classified as unknown-tcp or unknown-udp.

When a recognizable and sufficiently unique application pattern is available, administrators can create a custom App-ID signature to identify the traffic.

How Does App-ID Work?

When traffic traverses a Palo Alto Networks firewall, App-ID analyzes the communication to identify the application and enable application-based Security Policy enforcement.

Application identification is progressive and may change as additional packets are exchanged.

The general identification process is as follows:

  1. Initial Session Processing: The firewall receives traffic and performs initial session and Security Policy processing based on the information available.

  2. Application Signature Matching: App-ID examines available traffic characteristics and evaluates applicable application signatures.

  3. Protocol Decoding: Supported protocol decoders interpret the application protocol and provide contexts for identifying applications within the traffic.

  4. Heuristic Analysis: Additional identification techniques may be used when application signatures and protocol decoding alone are insufficient.

  5. Application Identification: The firewall identifies the application based on the available evidence. Identification may become more specific as the session progresses.

  6. Security Policy Enforcement: PAN-OS evaluates the identified application against the configured Security Policies and applies the relevant policy action and supported security inspection.

  7. Logging and Visibility: Application information is available through the Traffic Logs and other monitoring features in PAN-OS.

Application Identification and Service Ports

App-ID is designed to identify applications independently of their commonly used ports. However, the service configured in the Security Policy determines which destination ports are permitted.

For applications communicating on their defined default ports, the recommended configuration is generally application-default.

If an application must communicate over a nonstandard port, administrators should review whether an explicitly defined service is required.

For custom applications, the appropriate default port can be configured within the custom application definition.

Application Identification and Encrypted Traffic

When traffic is encrypted using SSL/TLS or SSH, App-ID may still identify certain applications using visible protocol information and traffic characteristics.

However, if the application signature requires access to encrypted payload data, decryption may be necessary.

Without decryption, the firewall may identify the traffic only as a generic encrypted application, such as ssl or ssh, or as another application identifiable from the available information.

Note: Decryption is not mandatory for every App-ID identification. Its necessity depends on the application protocol, available metadata, and the information required for identification.

How Does PAN-OS Handle Unknown Applications?

During App-ID adoption or Security Policy optimization, administrators may encounter traffic that PAN-OS cannot identify.

Unknown applications should be investigated to determine whether they represent legitimate business applications, unsupported protocols, incomplete sessions, or potentially unwanted traffic.

PAN-OS commonly uses the following classifications:

ClassificationDescription
unknown-tcpTCP traffic for which the application could not be identified.
unknown-udpUDP traffic for which the application could not be identified.
incompleteTCP traffic where a complete three-way handshake was not observed.
insufficient-dataTraffic where insufficient application data was available to determine the application.

Incomplete Sessions

The incomplete classification generally indicates that the firewall did not observe a complete TCP three-way handshake.

Possible causes include:

  • The server does not respond to the client's SYN.

  • The TCP handshake is interrupted.

  • Asymmetric routing prevents the firewall from observing the complete handshake.

  • The session terminates before the handshake completes.

For these sessions, investigate the network connectivity and TCP handshake before attempting to create a custom application signature.

Insufficient Data

The insufficient-data classification indicates that the firewall did not receive enough application data to identify the application.

For example, a TCP connection may complete successfully but exchange little or no application payload before terminating.

Possible causes include:

  • The application closes the connection before exchanging sufficient data.

  • The session contains limited application payload.

  • The application communicates using an uncommon or minimal protocol exchange.

Unknown TCP and UDP Applications

Traffic classified as unknown-tcp or unknown-udp may contain application data that PAN-OS cannot associate with an existing App-ID.

Possible reasons include:

  • Proprietary or internally developed applications.

  • Unsupported application protocols.

  • Application payloads that do not match existing signatures.

  • Uncommon protocol behavior.

  • Encrypted traffic with insufficient visible identification information.

In these scenarios, administrators should review the Traffic Logs and collect packet captures to determine whether a unique application signature can be created.

Should Application Override Be Used?

Application Override is not required to investigate unknown applications and should not be considered the default solution.

Application Override allows administrators to assign a predefined application classification to traffic matching specified criteria without normal App-ID identification.

However, traffic matching an Application Override policy bypasses normal App-ID processing, which can limit application-layer threat inspection.

When application identification and Layer 7 inspection are required, creating a custom application signature is generally preferred.

How to Investigate Unknown Applications

Before creating a custom App-ID signature, administrators should determine why the application is not being identified.

Step 1: Review the Traffic Logs

Navigate to:

Monitor > Logs > Traffic

Filter the relevant session using the source IP address, destination IP address, application, or port.

Review the following information:

  • Identified application

  • Source and destination addresses

  • Source and destination ports

  • Security Policy rule

  • Session end reason

  • Bytes and packets exchanged

  • Action taken by the firewall

Step 2: Verify the Connection

For TCP traffic, confirm that the three-way handshake completes successfully.

If the session is classified as incomplete, investigate connectivity or routing before proceeding with App-ID signature analysis.

Step 3: Verify Application Data

Determine whether the client and server exchange application payload.

If the TCP handshake completes but insufficient application data is exchanged, the firewall may classify the session as insufficient-data.

If application payload is present but the firewall cannot identify the protocol, further packet analysis may be required.

Step 4: Collect Packet Captures

Collect packet captures that include the initial connection establishment and the application data exchanged between the client and server.

Whenever possible, collect multiple sessions representing normal application operations.

These captures help identify application-specific patterns that can be used to create custom signatures.

How to Create a PCAP

A packet capture (PCAP) records network packets and allows administrators to analyze the communication between a client and server.

For App-ID investigations, a PCAP can help determine:

  • Whether the TCP handshake completes.

  • Whether application data is exchanged.

  • Which protocol is being used.

  • Whether the application payload is encrypted.

  • Whether a unique and consistent application pattern is present.

Collect a PCAP Using the PAN-OS Web Interface

  1. Navigate to Monitor > Packet Capture.

  2. Configure a packet filter for the relevant source and destination addresses.

  3. Specify the protocol or port where necessary.

  4. Enable filtering.

  5. Configure the required packet capture stages.

  6. Enable packet capture.

  7. Reproduce the application traffic.

  8. Disable packet capture.

  9. Download the PCAP files for analysis.

The available packet capture stages include:

Capture StageDescription
ReceiveCaptures packets entering the firewall dataplane.
FirewallCaptures packets at the firewall processing stage.
TransmitCaptures packets transmitted by the firewall.
DropCaptures packets dropped by the firewall.

For additional information, refer to:

How to Run a Packet Capture

Note: Packet capture should be enabled only for the required troubleshooting period. Use restrictive filters to minimize resource consumption and avoid collecting unrelated traffic.

How to Analyze the PCAP

Once the PCAP has been collected, open it using Wireshark or another supported packet analysis tool.

Identify the relevant TCP or UDP conversation.

For TCP-based applications:

  1. Locate the relevant application session.

  2. Verify the TCP handshake.

  3. Identify packets containing application data.

  4. Right-click a relevant packet.

  5. Select Follow > TCP Stream.

  6. Examine the application data exchanged between the client and server.

For HTTP-based applications, review request methods, URI paths, headers, and other application-specific fields.

For proprietary TCP applications, look for recognizable byte sequences, protocol identifiers, message headers, or other consistent patterns.

The selected pattern should:

  • Be unique to the application.

  • Remain consistent across multiple sessions.

  • Be present during normal application communication.

  • Avoid dynamic values such as IP addresses, TCP sequence numbers, timestamps, or ephemeral ports.

  • Be sufficiently specific to minimize false-positive identification.

Note: More than one application session should be analyzed before selecting a signature pattern. Testing against unrelated application traffic is also recommended.

How to Create a Custom App-ID

Once a unique and consistent application pattern has been identified, administrators can create a custom application and associate a signature with it.

Step 1: Create the Custom Application

Navigate to:

Objects > Applications > Add

Configure the relevant application properties, including:

  • Application name

  • Category and subcategory

  • Technology

  • Risk level

  • Application characteristics

  • Parent application, if applicable

  • Default ports

  • Session timeout settings, where required

The application properties should accurately represent the behavior and intended use of the custom application.

Step 2: Create the Application Signature

Open the Signatures tab and select Add.

Configure the signature name, scope, and matching conditions.

Select the appropriate signature context based on the application protocol.

For example, an HTTP-based application may use an HTTP request URI context, while a proprietary TCP application may require a supported TCP payload context.

Configure a pattern that uniquely identifies the application.

Additional matching conditions may be used to improve accuracy and reduce false positives.

Important: Custom App-ID signatures support pattern matching using PAN-OS-supported regular-expression syntax. Hexadecimal notation may be used to represent binary data, but hexadecimal format is not mandatory for every signature.

The selected pattern must comply with the signature syntax and context requirements supported by the PAN-OS version.

Step 3: Commit the Configuration

Save the custom application and commit the configuration.

Step 4: Configure the Security Policy

Navigate to:

Policies > Security

Create or update the Security Policy to allow the custom application where required.

Configure:

  • Appropriate source and destination zones.

  • Required source and destination addresses.

  • Custom application in the Application field.

  • Appropriate service configuration.

  • Required security profiles.

  • Logging at session end.

When using application-default, ensure the custom application has the correct default port configuration.

Review application dependencies and any initial application classifications required for the session to reach the custom App-ID identification stage.

Step 5: Verify Application Identification

Generate new application traffic and navigate to:

Monitor > Logs > Traffic

Confirm that the firewall identifies the traffic using the newly created custom App-ID.

Verify that:

  • The correct application is identified.

  • The expected Security Policy is matched.

  • The application operates as expected.

  • Supported security inspection remains available.

  • Unrelated applications are not incorrectly identified.

If the custom application is not identified, review the signature pattern, selected context, parent application, decryption requirements, and application payload.

Best Practices

  • Investigate unknown applications before creating custom signatures.

  • Verify TCP connectivity before troubleshooting application identification.

  • Collect multiple PCAPs to identify consistent application characteristics.

  • Prefer protocol-specific signature contexts when appropriate.

  • Avoid matching dynamic packet fields or common protocol strings.

  • Use multiple signature conditions where necessary to improve accuracy.

  • Review decryption requirements for encrypted applications.

  • Avoid unnecessary Application Override policies.

  • Verify application dependencies and Security Policy configuration.

  • Validate custom signatures before deploying them in production.

  • Reevaluate custom signatures when application behavior changes.

See Also

 



Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000CloPCAS&lang=en_US&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail

Choose Language