Panorama Unresponsive or Cannot Log In After PAN-OS Upgrade or Reboot

Issue

• Panorama is Unresponsive or you cannot log in After PAN-OS Upgrade or Reboot
• The regular credentials local/other authentication do not work on Panorama.
• 'Invalid credentials' message is returned.
• The password is confirmed to be correct.

Cause

The system is doing a fsck (file system integrity check) on the panlogs partition. This will happen every 4 reboots for single mount and 8 reboots for Dual Mount Disks or 90 days since the last fsck was performed, whichever first.  Depending on the size of the partition, the fsck may take a while.These settings are intentional and cannot be bypassed.

Detail

Here are the logs that show the start and end of that process being launched and ended.

mp\masterd_detail.log              08-07 14:39:52   DEBUG: dsms: Adding
external script /sbin/new-panlogs-check /var/log/pan/panlogs-partition.log,
count 1, event startScriptsDone
mp\masterd_detail.log              08-07 15:29:57   DEBUG: dsms: Got event
startScriptsDone

If the user is on the console, (s)he will see 'wall' (write all) messages to the effect that the fsck is happening.

Note: The admin account (all accounts) is added by the mgmtsrvr process which can't be started until panlogs are mounted.  However, panlogs cannot be mounted until a fsck is done.  The admin account isn't added until after this operation is completed and mgmtsrvr is allowed to run on the mounted panlogs.

Resolution

Wait for 30 min ~ couple of hours or more for the FSCK to be complete in order to login.

owner: jdavis