HA failover triggerd due to "Out of memory condition detected, kill process".
3275
Created On 06/05/25 22:34 PM - Last Modified 12/29/25 19:35 PM
Symptom
- System logs shows OOM logs for multiple processes : "2025/04/03 07:55:49 critical general general 0 Out of memory condition detected, kill process 9809"
- Core files are generated related to all_task after the crash.
- Device has large sdwan configuration.
- Slow memory leak is observed for log-receiver process with subsequent commits.
xxxx/xx/xx xx:xx:22 processes Name PID CPU% FDs Open Virt Mem Res+Swap State Res+Swap-Lazy
xxxx/04/14 13:10:59 processes logrcvr 6128 0 130 3083824 1313496 S 856480
xxxx/04/14 13:13:59 processes logrcvr 6128 0 130 3083824 1320280 S 863052
xxxx/04/14 13:18:01 processes logrcvr 6128 0 130 3083824 1311776 S 856124
xxxx/04/15 13:11:28 processes logrcvr 6128 0 125 3145264 1360920 S 897700
xxxx/04/15 13:14:28 processes logrcvr 6128 0 125 3145264 1364568 S 901360
xxxx/04/15 13:18:33 processes logrcvr 6128 0 125 3145264 1359412 S 900720
xxxx/04/16 13:11:47 processes logrcvr 6128 0 125 3186224 1389044 S 938068
xxxx/04/16 13:14:49 processes logrcvr 6128 0 125 3186224 1389044 S 937804
xxxx/04/16 13:18:57 processes logrcvr 6128 0 125 3186224 1390812 S 943696
Environment
- Palo Alto Networks Firewalls
- Supported PAN-OS
- SDWAN
Cause
SDWAN config reading in log-receiver causing leaks in the memory with subsequent commits.
Resolution
Resolution:
- The issue will be resolved under PAN-287314 in the upcoming versions.
- Upgrade the PANOS to any of the fixed versions when released.
- 11.2.8
- 12.1.2
- 12.2.0
- 11.1.11
- 10.2.17
- 10.2.9-h24
Workaround:
- Restart Log Receiver from CLI by running, "debug software restart process log-receiver"
- Frequency of w/a execution depends directly of the amount of memory being leak and will vary on case by case.
Additional Information