Google Stackdriver does not receive metrics from PA-VM on GCP due to DNS resolution issue

• Firewall is configured to publish PAN-OS metrics to Google Stackdriver (default interval is 5 minutes).
• The requirement is to monitor Active Session and DP CPU utilisation.
• After 5 minutes, the firewall should  start sending metrics to GCP.
• This does not happen. When navigating to GCP Monitoring > Metrics explorer > VM Instance >  Custom > custom/VMseries/panSessionActive and custom/VMseries/DataPlaneCPUUtilizationPct, there is no value displayed on the graph.
• "Temporary failure in name resolution" is seen in the plugin log.

> less plugins-log pan_vm_plugin.log
http://metadata.google.internal/computeMetadata/v1/instance/ attributes/serial-port-enable from GCP
+0000 vm_cloudwatch_push_metrics INFO: : Getting metadata failed with error <urlopen error [Errno -3] Temporary failure in name resolution>

• Firewall lookup for FQDN metadata.google.internal fails and is not resolved.

> ping host metadata.google.internal
ping: unknown host metadata.google.internal

• PA-VM on Google Cloud Platform (GCP)
• Supported PAN-OS version
• Google Stackdriver configured to monitor metrics from PA-VM

• Firewall is configured with a DNS server that is failing to resolve FQDN metadata.google.internal.
• In this example, the firewall is configured to use DNS server 10.20.2.20.

1. Remove DNS server configured on the firewall. This will cause the firewall to use default GCP DNS.
2. Alternatively, use another DNS server that can resolve FQDN metadata.google.internal.
3. Commit the configuration changes.
4. After the change, Wait for at least 5 minutes before refreshing the GCP Monitoring > Metrics explorer to verify values of panSessionActive and DataPlaneCPUUtilizationPct.

• After the change one can Verify if the firewall is able to resolve to FQDN metadata.google.internal

> ping host metadata.google.internal
PING metadata.google.internal (169.254.169.254) 56(84) bytes of data.
64 bytes from metadata.google.internal (169.254.169.254): icmp_seq=1 ttl=255 time=0.075 ms
....

• Similarly plugin log should report "Getting metadata succeeded"

> less plugins-log pan_vm_plugin.log
.....
+0000 post_commit INFO: : Getting metadata succeeded
+0000 vm_cloudwatch_push_metrics INFO: : vm_mode: 7
+0000 vm_cloudwatch_push_metrics INFO: : Platform Identified as GCP
+0000 vm_cloudwatch_push_metrics INFO: : GCP cloud_setting called
+0000 vm_cloudwatch_push_metrics INFO: : Getting metadata http://metadata.google.internal/computeMetadata/v1/instance/ attributes/serial-port-enable from GCP
+0000 vm_cloudwatch_push_metrics INFO: : Getting metadata succeeded