VMware ESXi Panorama vm keeps crashing and log collector shows disconnected

VMware ESXi Panorama vm keeps crashing and log collector shows disconnected

17796
Created On 07/17/20 21:48 PM - Last Modified 10/21/20 02:14 AM


Symptom


VMware ESXi Panorama vm keeps crashing and log collector shows disconnected

Environment


VMware ESXi panorama vm appliance

Cause


High memory utilization by any one of the processes --> elasticsearch, java, esmonitoring.  Overall system swap memory utilization at 100%

Resolution


  1.  Check ESXi vCenter for snapshot.  If it is enabled, disable it snapshot on both active and (if applicable) passive panorama. 
  2. To disable snapshots, go to VM > Settings > Options > Snapshots.
  3. If no snapshots exist for the selected virtual machine, disable snapshot functionality by selecting Disable snapshot  
  4. If one or more snapshots exist for the selected virtual machine, the Disable snapshots option is disabled. If you want to disable snapshot functionality, you must first delete all snapshots for the current virtual machine. Refer to Deleting a Snapshot
  5. Run CLI command to check if any logging rate is incrementing.  
    > debug log-collector log-collection-stats show incoming-logs
  6. Run the following command to check logging rate 
    > debug software restart process logd
  7. If LC shows disconnected, push config out to LC group.  NOTE: LC should show connected after config push completes
  8.  Increase panorama vm memory from to up to 32 GB or higher as needed. 
  9. Run the following command to increase the jvm heap memory size to 12 GB or higher as needed. 
> debug logdb set-heap-size heap-size 12
 
  1.  Rebooted Panorama.
  2.  Log should start showing up from earlier timestamps to near real time.
 Run the following command to see LC log statistics beginning to refresh. 
> show system resource follow

NOTE:
CLI command should show 1-minute load back down to below 2.  


Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u0000008Uu9CAE&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail

Choose Language