How to troubleshoot firewall or Panorama log queue issues

1. For syslog server, refer to How To Troubleshoot Connection Failures To Syslog Servers.
2. For SNMP manager, refer to Forward Traps to an SNMP Manager.
3. For email server, use the Test Connection button within the email server profile to verify that the firewall can successfully authenticate with the email server. Refer to How to Send a Test Email to Verify Email Profile Settings.

> debug log-receiver statistics 

1. Check the external forwarding stats, look for the Drop Count, and see if that is incrementing:

   External Forwarding stats:
             Type  Enqueue Count     Send Count     Drop Count    Queue Depth     Send Rate(last 1min)
           syslog        9187198        9166138        3565042          16366                     6701
             snmp              0              0              0              0                        0
            email              0              0              0              0                        0
              raw              0       12747547              0              0                    12451
             http              0              0              0              0                        0
          autotag              0              0              0              0                        0
       quarantine              0              0              0              0                        0

2. For more details about which logs are dropping check the external forwarding stats per log type for the affected external log server:

   External Forwarding stats per log type for syslog:
             Type   Enqueue Count      Send Count      Drop Count
          traffic         9037714         9017508         3438854
           config               4               4               4
           system           21170           21155            3318
           threat          128310          127471          122866
         hipmatch               0               0               0
           userid               0               0               0
            iptag               0               0               0
          extpcap               0               0               0
              gtp               0               0               0
             auth               0               0               0
             sctp               0               0               0
    globalprotect               0               0               0
       decryption               0               0               0

> debug log-collector log-collection-stats show log-forwarding-stats

1. Check the counters related to the affected log server, and look for the dropped count. See if that is incrementing. Example:

   syslog enqueued count: 150612012
   syslog sent count: 150349867
   syslog dropped count: 189121069
   syslog Queue depth: 262144

> debug software process restart log-receiver

1. For a Panorama-managed firewall, navigate to Panorama > Managed Devices > Health in the Panorama UI. This will allow you to check the firewall resources and its logging rate. Click on the affected firewall's name then on the Logging Tab for the detailed Logging Rate graph and the Resources Tab for the detailed Memory, CPU, Packet Buffers, and Packet Descriptors graphs.
2. 

1. For Syslog Server:
   

   1. Check if Panorama is hitting any SW issues, like: PAN-257615 which fixed an issue on Panorama where logs did not display or displayed intermittently on the web interface.

   2. Check if the firewall or Panorama is hitting SW issue: PAN-234929 that fixed an issue where Fixed an issue where tabs in the ACC like Network Activity and Threat Activity may not display data correctly within certain time filters.

2. SNMP Manager:
   1. Ensure the SNMP manager is correctly ingesting logs. If log ingestion is slow, it may create additional overhead for logrcvr (due to queuing and hint mechanisms), which can impact the log forwarding performance of the firewall or Panorama. Logs intended for forwarding to the SNMP manager are typically dropped if the connection to the SNMP server is broken or unstable, or if the SNMP server is unable to acknowledge the logs promptly.
3. Email Server and HTTP server:
   1. Email server and HTTP log forwarding are designed for occasional notifications, not high-volume logs like Syslog. Ensure these services are not configured for heavy log traffic.