PA-VM deployed in AWS does not move ENI’s to newly active unit upon HA failover due to unsynchronised time

PA-VM deployed in AWS does not move ENI’s to newly active unit upon HA failover due to unsynchronised time

1791
Created On 06/21/23 04:53 AM - Last Modified 03/27/24 20:41 PM


Symptom


  • Upon HA failover, the newly active firewall instance cannot pass traffic as dataplane interfaces are down.
  • On the AWS portal, Elastic Network Interfaces (ENI’s) did not get transferred to newly active firewall instance.
  • Plugin logs on the newly active firewall indicate API calls made by the VM-Series plugin to AWS EC2 services are failing
> less mp-log pan_vm_plugin.log
+0700 vm_ha_state_trans INFO: : AWS vm_ha_trans called
+0700 vm_ha_state_trans INFO: : AWS get_meta_data called http://169.254.169.254/latest/ meta-data/instance-id
2023-05-26 22:35:42.873 +0700 vm_ha_state_trans INFO: : AWS get_meta_data succeeded
+0700 vm_ha_state_trans INFO: : Local instance:i-095c5a11b86c2ea5a Remote instance:i-04f41d74e42fa8d32
+0700 vm_ha_state_trans INFO: : EC2 get interface info failed for instance-i-04f41d74e42fa8d32
An error occurred (AuthFailure) when calling the DescribeNetworkInterfaces operation: AWS was not able to validate the provided access credentials


Environment


  • Platform: PA-VM
  • PAN-OS / Plugin Version: 9.1.x or newer / 2.0.x or newer
  • Deployment: AWS


Cause


Firewall date and time is out of sync. AWS denies API calls made with unsynced timestamp.

Resolution


Verify if the following are correct.
  1. The Firewall's management interface has internet access.
  2. Firewall can resolve DNS names.
  3. IAM Role has sufficient permissions.
Once verified, Configure NTP on both firewall peers:
  1. Login to firewall GUI.
  2. Go to Device > Setup > Services.
  3. Click on the Edit button.
  4. Click on the NTP tab.
  5. In the NTP Server Address field, enter the IP address or hostname of a NTP server.
  6. Click OK.
  7. Commit the changes.
Note: You can use AWS default NTP server 169.254.169.123, or another valid NTP server.


Actions
  • Print
  • Copy Link

    https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u000000kI9ECAU&lang=en_US&refURL=http%3A%2F%2Fknowledgebase.paloaltonetworks.com%2FKCSArticleDetail