“Failed to get Azure Access Token” error shows up in plugin logs when floating IPs on one firewall in Azure do not move to the secondary firewall after failover
Symptom
When two VM firewalls are configured in High Availability Active/Passive configuration in HA for Secondary-IP move, secondary IPs do not move to the passive device from Active device even though HA state change takes place successfully
Environment
- Two VM firewalls with same PAN-OS versions, plugin versions and all the licenses alike
- Issue takes place on any PAN-OS version
- Both the firewalls should be in Active/Passive HA configuration
Cause
It needs to be made sure that proper IAM role is assigned to the right service principal and that App registration is done for the right subscription within Azure and assigned to the right NIC. Following errors show up in vm_plugin.log during this kind of an issue as under:
2022-12-05 02:43:13.144 -0800 vm_ha_state_trans INFO: : Failed to retrieve response from resource manager endpoint
2022-12-05 02:43:13.145 -0800 vm_ha_state_trans INFO: : Failed to get Azure Access Token
2022-12-05 02:43:17.343 -0800 vm_ha_state_trans INFO: : vm_mode: 6
2022-12-05 02:43:17.538 -0800 vm_ha_state_trans INFO: : Platform Identified as AZR
2022-12-05 02:43:17.739 -0800 vm_ha_state_trans INFO: : vm_get_ht_perf_opt called
2022-12-05 02:43:17.766 -0800 vm_ha_state_trans INFO: : syst count: 4
2022-12-05 02:43:17.767 -0800 vm_ha_state_trans INFO: : AZR cloud_setting called
2022-12-05 02:43:17.947 -0800 vm_ha_state_trans INFO: : AZR vm_ha_trans called
2022-12-05 02:43:17.947 -0800 vm_ha_state_trans INFO: : resource_mgr_endpoint success
2022-12-05 02:43:18.019 -0800 vm_ha_state_trans INFO: : Exception caught in pycurl_request: (77, 'error setting certificate verify locations:\n CAfile: /opt/pancfg/mgmt/plugins/appdata/cacert.pem\n CApath: none'), error code: 0
2022-12-05 02:43:18.019 -0800 vm_ha_state_trans INFO: : AZR set_endpoint_URL failed to find URL from response, err: 'NoneType' object has no attribute '__getitem__'
Resolution
- Make sure Role assignment is done properly for the correct service principal in Azure and App registration should be properly done
- Customers should give at least “Contributor” Role
- Role assignments should be done for the right subscription and should be assigned to the right NIC.https://learn.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal
- Engage Azure TAC if necessary to verify Role assignment on Azure end