HA failover triggerd due to "Out of memory condition detected, kill process".

HA failover triggerd due to "Out of memory condition detected, kill process".

1828
Created On 06/05/25 22:34 PM - Last Modified 08/19/25 07:13 AM


Symptom


  • System logs shows OOM logs for multiple processes : "2025/04/03 07:55:49 critical general general 0 Out of memory condition detected, kill process 9809"
  • Core files are generated related to all_task after the crash.
  • Device has large sdwan configuration.
  • Slow memory leak is observed for log-receiver process with subsequent commits.
xxxx/xx/xx xx:xx:22 processes Name                   PID     CPU% FDs Open     Virt Mem     Res+Swap   State      Res+Swap-Lazy 
xxxx/04/14 13:10:59 processes logrcvr                6128     0     130        3083824      1313496      S          856480

xxxx/04/14 13:13:59 processes logrcvr                6128     0     130        3083824      1320280      S          863052
xxxx/04/14 13:18:01 processes logrcvr                6128     0     130        3083824      1311776      S          856124
xxxx/04/15 13:11:28 processes logrcvr                6128     0     125        3145264      1360920      S          897700
xxxx/04/15 13:14:28 processes logrcvr                6128     0     125        3145264      1364568      S          901360
xxxx/04/15 13:18:33 processes logrcvr                6128     0     125        3145264      1359412      S          900720
xxxx/04/16 13:11:47 processes logrcvr                6128     0     125        3186224      1389044      S          938068
xxxx/04/16 13:14:49 processes logrcvr                6128     0     125        3186224      1389044      S          937804
xxxx/04/16 13:18:57 processes logrcvr                6128     0     125        3186224      1390812      S          943696


Environment


  • Palo Alto Firewalls
  • Supported PANOS
  • SDWAN
  • Commit


Cause


SDWAN config reading in log-receiver causing leaks in the memory with subsequent commits.



Resolution


Resolution:

  1. The issue will be resolved under PAN-87314 in the upcoming versions.
  2. Upgrade the PANOS to any of the fixed versions when released.
    • 11.2.8
    • 12.1.2
    • 12.2.0
    • 11.1.11
    • 10.2.17
    • 10.2.9-h24

Workaround:

  1. Restart Log Receiver from CLI by running, "debug software restart process log-receiver"
  2. Frequency of w/a execution depends directly of the amount of memory being leak and will vary on case by case.


Additional Information


 

 

 



Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA1Ki000000TNKHKA4&lang=en_US&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail