VM-Series Network Interface Enumeration Order Issue Following Azure Control Plane Migration
Symptom
- VM-Series firewalls with three or more NICs experience interface reordering inside PAN-OS after being stopped (deallocated) and started, or upon new deployment.
- Network traffic fails, routing breaks, or security management policies fail to apply correctly.
- Note: The interface order shown in the Azure Portal and via Azure APIs remains unchanged; the issue is observed exclusively inside the guest OS (PAN-OS).
How to Verify
- Note the MAC addresses of the NICs attached to the VM-Series firewall from the Azure portal or by using the Azure CLI.
- Verify the MAC addresses associated with each interface on the VM-Series firewall using the following PAN-OS CLI command:
> debug show vm-series interfaces all - Compare the interface-to-MAC address mapping in Azure with the interface-to-MAC address mapping shown in the PAN-OS CLI output.
- If the MAC addresses are found to be interchanged between interfaces, the firewall is impacted by the recent Azure control plane upgrade.
Environment
Product: VM-Series / AIRS
OS Type: PAN-OS
Version: Any
Topology/Configuration
VM-Series firewalls configured with three or more network interfaces (one management interface and at least two dataplane interfaces).
Cause
Microsoft Azure has been migrating the Virtual Machines (VMs) to a newer control plane infrastructure. During this migration, the underlying hardware layer enumeration process for guest OS network interfaces changed. While Azure APIs and Portal representations retained the original sequence, the internal guest OS sequence shifted for multi-NIC configurations (3+ NICs).
Azure does not natively guarantee guest-side interface ordering. Palo Alto Networks virtual firewall maintains interface order inside its operating system (PAN-OS) using MAC addresses. However, in certain cases like new deployments or network interface additions/removal, the interfaces may be reordered within PAN-OS, leading to operational disruptions.
Resolution
Azure has halted the migration and creation of VMs on the new control plane infrastructure (outside of pre-production Canary and Pilot regions).
To revert to the previous interface ordering behavior, perform the following steps:
- Redeploy the Instance:
- Redeploy the VM instance via the Azure Portal or Azure CLI.
- Recreate Interface Mapping in PAN-OS:
- Power down (deallocate) the VM instance completely.
- Detach all secondary (data plane) interfaces from the VM.
- Start the VM instance.
- Wait for PAN-OS to complete the boot process.
- To confirm, log in and verify that the auto-commit job has completed successfully.
- Power down (deallocate) the VM instance again.
- Reattach the previously detached secondary (data plane) interfaces in the required order.
- Start the VM instance.
Additional Information
- Long-Term Plan: Microsoft is collaborating directly with Palo Alto Networks to refine network interface API behaviors and guest OS enumeration mechanisms to prevent similar issues in future platform updates.
- Scope: New deployments or Stop/Start actions for Network Virtual Appliances will no longer land on the impacted new control plane infrastructure.
- VM-Series firewalls deployed as part of a Virtual Machine Scale Set (VMSS) are not expected to be impacted.