Device telemetry streaming intermittently fails due to a bundle generation failure.

• The device cannot generate and send the telemetry bundle at the configured interval.
• In the firewall UI, under Device > Setup > Management >Telemetry, the three categories "Device Health and Performance", "Product Usage", and "Threat Prevention"  will be stuck on an earlier date, with the number of failed attempts showing as zero since then.

• There will be no new system logs indicating either the successful or unsuccessful sending of logs since that time.

• The following logs are observed in the device_telemetry_hour.log, indicating that the creation process for the telemetry tar file was stuck:

  2024-03-08 09:32:07,768 dt_hour INFO CreateTarFile: output_dir: /opt/panlogs/tmp/device_telemetry/_hour_1709911807.3035285 
  2024-03-08 09:33:03,919 dt_hour INFO DT - enabled !

  The expected logs for the telemetry tar file creation are missing.
• In a working scenario, the following logs would typically be seen:

  dt_hour INFO CreateTarFile: output_dir: /opt/panlogs/tmp/device_telemetry/_hour_1718073004.6149096
  dt_hour INFO tar cmd: /bin/tar --ignore-failed-read -czf 
  /opt/panlogs/tmp/device_telemetry/CURRENT_hour/PA_016201035889_dt_10.2.8-h4_20240611_0230_telemetry_data.tar.gz /tmp/DT/logs 
  /opt/panlogs/tmp/device_telemetry/_hour_1718073004.6149096 /opt/panlogs/tmp/device_telemetry/minute/

• NGFW
• Panorama
• Telemetry

This behavior is caused by a software issue identified as PAN-250062, where device telemetry might fail at configured intervals due to problems with bundle generation. This issue is targeted to be resolved in PAN-OS versions 10.2.8-h5, 10.2.9-h9(released and contain the fix), 10.2.10-h3(released and contain the fix), 10.2.11(released and contain the fix), 11.0.7, 11.1.5(released and contain the fix), 11.2.3(released and contain the fix), and later.

• PAN-OS 10.2.11 Addressed Issues
• PAN-OS 10.2.9-h9 Addressed Issues
• PAN-OS 11.1.5 Addressed Issues
• PAN-OS 11.2.3 Addressed Issues

Upgrade to a recommended PAN-OS version that is equal to or higher than the release containing the fix for the software issue.
 

As of today 12/18/2024 only 10.2.9-h9, 10.2.10-h3, 10.2.11, 11.1.5 and 11.2.3 PAN-OS versions which carry the fix have been released.

IMPORTANT: A new software issue affecting the sending of telemetry data and impacting 10.2.12-h1 has been found. Refer to PAN-272413.

PAN-OS 10.2.13-h1 Addressed Issues.
PAN-OS 10.2.12-h3 Addressed Issues.