How to Troubleshoot Slow Report Job

How to Troubleshoot Slow Report Job

397
Created On 09/15/25 17:07 PM - Last Modified 09/15/25 17:33 PM


Objective


  • Troubleshoot slow report jobs.

Environment


  • Strata Cloud Manager
  • Slow Report Jobs

Procedure


  1. Identifying slow jobs: You can use the following CLI command to list all running and recently completed reports. The output is useful for finding the report's ID and seeing how long it has been running:  
    show report jobs

    Note: on Panorama, this command is not useful for reports being run on a managed firewall; you must run the command on the firewall itself.

  2. Common Causes for Slow Reports: Slow report generation can often be linked to high Management Plane (MP) CPU utilization. Key causes include:
    1. Concurrent Jobs: Running multiple demanding processes simultaneously can slow down the system. This includes running reports during commits, multiple administrators being logged in, or collecting tech support files while other jobs are active.
    2. Report Configuration:
      1. A high number of scheduled reports.
      2. Complex reports with many "sorted by" or "grouped by" criteria.
      3. A long time range for the reporting period.
    3. Sequential Processing: Scheduled reports on a firewall run one after another. If one report takes a long time to complete, all other reports in the queue must wait, which can cause significant delays.
  3. Troubleshooting Steps: When a report job is slow, collecting data can be difficult as it may add further load to the MP CPU.
    However, the following steps can help diagnose the issue in real time:
    1. Check System Resources:  To get a real-time view of MP resources and identify which processes are consuming the most CPU, use the CLI command: 
      show system resources follow
    2. View All Jobs:  To see everything currently running on the device, run the CLI command: 
      show jobs all
    3. Review Logs:
      • Check the masterd_detail.log to see if the report generation process (pan_report_gen) was scheduled and executed correctly. This is particularly useful if scheduled reports have stopped appearing altogether.
      • Review mp-monitor.log files for detailed performance information.
  4. Resolution and Best Practices: To prevent slow report jobs, consider the following best practices:

    1. Offload Reporting: The most effective solution is to forward logs to a Panorama and run all reports from there instead of on the firewall itself. 

    2. Optimize Reports: 

      1. Reduce the total number of custom and scheduled reports. 

      2. Simplify reports by reducing the number of sorting/grouping conditions and shortening the time period they cover. 

      3. Disable predefined reports if they are not needed. 

    3. Schedule Jobs Efficiently:

      1. Avoid scheduling periodic events like content updates, log exports, and reports at the same time. Space them at least 30-45 minutes apart.

      2. Do not schedule background jobs to run between 2:00 AM and 2:45 AM, which is a common default time for daily report execution.

    4. Reduce System Load:

      1. Minimize administrative logins during peak business hours, as UI refreshes consume resources.

      2. Change the GUI refresh interval to manual.

      3. Optimize logging by removing it from rules for non-essential traffic like DNS, NetBIOS, or internal trusted traffic.
Other users also viewed:
Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA1Ki000000TNuFKAW&lang=en_US&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail