Unable to delete Azure VM Extensions or convert from Managed disk to Unmanaged disk
45723
Created On 07/10/20 18:06 PM - Last Modified 07/23/20 02:30 AM
Symptom
- Unable to convert managed to unmanged disk on Azure for VM-Series firewalls.
- Third party Microsoft Azure Extensions installed on VM-Series.
- Noticed "Not Ready" state on Linux waagent.
- Unable to uninstall Azure extension on portal.
- May try running Azure CLI force delete extension command from powershell. Refer: https://docs.microsoft.com/en-us/powershell/module/azurerm.compute/remove-azurermvmextension?view=azurermps-6.13.0
- But, force delete Extension from Azure CLI, may fail with error:
ErrorCode: VMAgentStatusCommunicationError
ErrorMessage: VM 'GabstaWGA' has not reported status for VM agent or extensions. Verify the VM has a
running VM agent and that it can establish outbound connections to Azure storage. Please refer to
https://aka.ms/vmextensionwindowstroubleshoot for additional VM agent troubleshooting information.
LogsUnder pan_vm_agent.log:
2020/03/07 22:55:26.875098 ERROR Reporting NotReady failed: [000008] [Wireserver Exception] [000009] None
2020/03/07 22:55:26.876673 ERROR Event: name=WALinuxAgent, op=Provision, message=[000008]
[Wireserver Exception] [000009] None, duration=0
2020/03/07 22:55:26.878322 ERROR Event: name=WALinuxAgent, op=Provision, message=[000008] [CopyOvfEnv] Error
mounting dvd: [000007] Failed to get dvd device from /dev
Environment
- Platform: VM-Series on Microsoft Azure
- PAN-OS / Plugin Version: Any
- Deployment: Existing
Cause
- Palo Alto VM does not support any Azure extensions to be installed on the PA-VM and services like waagent cannot be installed on the VM-Series.
- waagent version is tied to PAN-OS release and cannot be upgraded or downgraded.
- Also, waagent version cannot be restarted on PAN-OS:
[root@vmseries-active ~]# service waagent start
waagent: unrecognized service
[root@vmseries-active ~]# service walinuxagent start
walinuxagent: unrecognized service
- waagent version can be found under pan_vm_agent.log:
pan_vm_agent.log <omitted> INFO Installed Agent WALinuxAgent-2.2.16 is the most current agent
pan_vm_agent.log <omitted> INFO Agent WALinuxAgent-2.2.16 is running as the goal state agent
Resolution
Re-deploy VM-Series firewall on Azure using the steps mentioned below.
- Deploy PANW VM-Series in the same environment as source/old PA-VM. Such as for instance size, region, PAN-OS release, etc.
- Register this VM.
- Add it to Panorama and add this new PA-VM to the same DG/Template as original VM, if required.
- Export device-state or running configuration from source/old VM
- Attach a new ‘dummy’ NIC to the source/old VM
- De-allocate Source/old VM
- Remove all NIC’s from the source/old VM
- Attach all NIC’s to the new VM same as the original.
- Remove the NIC that gets created with the VM.
- Restore the configuration to the VM
You may also refer PANW Github repositories to deploy firewalls using ARM Templates or Terraform.
Additional Information
Microsoft articles to troubleshoot VM extensions:
- https://docs.microsoft.com/en-us/azure/virtual-machines/extensions/features-windows#troubleshoot-vm-extensions
- https://docs.microsoft.com/en-us/azure/backup/backup-azure-troubleshoot-vm-backup-fails-snapshot-timeout
- https://docs.microsoft.com/en-us/powershell/module/azurerm.compute/remove-azurermvmextension?view=azurermps-6.13.0