PAN-OS 5.0.1, 5.1, 6.0, 7.0, 7.1
Issue
A platform's management server keeps crashing and restarting when it exceeds its virtual limit capacity.
Error
System logs indicating the management server exceeding its virtual limits
2012/11/19 10:20:38critical general general 0 "mgmtsrvr - virtual memory limit exceeded, restarting"
2012/11/19 10:29:39critical general general 0 "mgmtsrvr - virtual memory limit exceeded, restarting"
2012/11/19 10:56:41critical general general 0 "mgmtsrvr - virtual memory limit exceeded, restarting"
masterd.log
Nov 19 10:20:19 INFO: mgmtsrvr: received user restart
Nov 19 10:20:19 INFO: mgmtsrvr: User restart reason - Virtual memory limit exceeded (2575744 > 2560000)
Nov 19 10:20:19 INFO: mgmtsrvr: received user stop
Nov 19 10:20:25 INFO: mgmtsrvr: exited, Core: False, Exit code: 0
Resolution
Increase the virtual limit size of the management server.
To check the virtual memory limit on the device, run the following command:
> debug software virt-limit service mgmtsrvr limit <0-4294967295>
To set the virtual memory limit to 4GB, run the following command:
> debug software virt-limit service mgmtsrvr limit 4000000
Note: A reboot is not required for the increased virtual memory to take effect.
Verify the setting change by running the following command:
> show system state | match virtLimitEnable
md.apps.s0.mp.prc.mgmtsrvr.add-event: virtLimitEnable 4000000
md.apps.s0.mp.prc.mgmtsrvr.script.runtime: { 'actions': [ ], 'count': 1, 'display': , 'done-actions': [ { 'action': hb-start, }, ], 'external-restart-ok': True, 'group': { }, 'hb-enable': True, 'limits': { 'enable-fd-limit': True, 'enable-virt-limit': True, 'fd-limit': 50000000, 'virt-limit': 4000000, }, 'process': { 'last-pid': -1, 'pid': 1512, }, 'restart-enable': True, 'state-machine': { 'count': 1, 'event': virtLimitEnable 4000000, 'state': running, }, 'sysd-namespaces': [ ], 'sysd-notifiers': { }, }
admin@Panorama-54>
To revert changes, run the following command:
> debug software no-virt-limit service mgmtsrvr
Confirm the with the following command:
> show system state | match virtLimitDisable
md.apps.s1.mp.prc.mgmtsrvr.script.runtime: { 'actions': [ ], 'count': 1, 'display': , 'done-actions': [ { 'action': hb-start, }, ], 'external-restart-ok': True, 'group': { }, 'hb-enable': True, 'limits': { 'enable-fd-limit': False, 'enable-virt-limit': False, 'fd-limit': 1024, 'virt-limit': 4000000, }, 'process': { 'last-pid': -1, 'pid': 2259, }, 'restart-enable': True, 'state-machine': { 'count': 1, 'event': virtLimitDisable, 'state': running, }, 'sysd-namespaces': [ ], 'sysd-notifiers': { }, }
owner: kadak